Skip to content

Add ScrapingDog API integration for LinkedIn job search#2

Merged
iliasaz merged 5 commits intomainfrom
feature/scrapingdog-integration
Jan 26, 2026
Merged

Add ScrapingDog API integration for LinkedIn job search#2
iliasaz merged 5 commits intomainfrom
feature/scrapingdog-integration

Conversation

@iliasaz
Copy link
Owner

@iliasaz iliasaz commented Jan 25, 2026

Summary

  • Add ScrapingDog API integration to enable direct LinkedIn job searches alongside existing GitHub-sourced job postings
  • Implement secure API key storage in Keychain with settings UI
  • Create actor-based service with rate limiting (1s between requests)
  • Add search UI with filters for location, job type, experience level, and work type

Changes

New Files

  • ScrapingDogModels.swift - Codable models for API request/response
  • ScrapingDogService.swift - Actor-based API service
  • ScrapingDogSearchView.swift - SwiftUI search controls
  • ScrapingDogTests.swift - Unit tests (18 tests)

Modified Files

  • KeychainService.swift - ScrapingDog API key storage
  • SettingsView.swift - API key input section
  • ContentView.swift - Search UI integration
  • DatabaseManager.swift - Migration 013 for source_type

Test plan

  • Build and run tests (xcodebuild test)
  • Open Settings, add ScrapingDog API key, verify save/load
  • Toggle LinkedIn Search in main view toolbar
  • Search for jobs with various filters
  • Verify results appear with "LinkedIn" as aggregator
  • Verify jobs persist after reload
  • Test error states (no API key, invalid key)
  • Verify existing GitHub jobs still work alongside LinkedIn jobs

🤖 Generated with Claude Code

iliasaz and others added 5 commits January 25, 2026 14:59
- Add ScrapingDog API key storage in Keychain
- Create ScrapingDogService actor for API communication with rate limiting
- Add search UI with filters for job type, experience level, work type
- Map ScrapingDog response to JobPosting model
- Store jobs via existing JobRepository for deduplication
- Add database migration for source_type tracking
- Add comprehensive unit tests for models and conversion

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Change endpoint from /linkedin/jobs to /jobs/
- Update job_type values: full_time, part_time, contract, etc.
- Update exp_level values: internship, entry_level, mid_senior_level, etc.
- Update work_type values: at_work, remote, hybrid
- Add tests for direct JSON array response format
- Remove unsupported internship job type (use exp_level instead)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add SourceFilter enum with All/LinkedIn/GitHub options
- Add source filter picker in the toolbar filter row
- Filter jobs by aggregatorName to show LinkedIn-only or GitHub-only

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add showGitHubSearch state variable
- Add GitHub button in toolbar with branch icon
- Hide GitHub URL input by default, show when button clicked
- Add green background tint to GitHub panel (matches LinkedIn's blue)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add has_easy_apply column to job_postings table (migration 014)
- Add hasEasyApply field to JobPosting and PersistedJobPosting models
- Add setEasyApply method to JobRepository
- Add LinkedIn rate limit setting in SettingsView (default 10 req/min)
- Modify JobAnalysisService to use ScrapingDog job details API for LinkedIn jobs
- Add rate-limited LinkedIn page fetching for Easy Apply detection
- Add Easy Apply indicator (bolt icon) in job table first column
- Detect Easy Apply via EASY_APPLY_DOCUMENT marker in LinkedIn HTML

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@iliasaz
Copy link
Owner Author

iliasaz commented Jan 26, 2026

Latest Updates (Easy Apply Detection)

Added the following enhancements:

New Features

  • Easy Apply Detection: LinkedIn jobs now display a ⚡ bolt icon if they support Easy Apply
  • ScrapingDog Job Details API: Fetches full job descriptions from ScrapingDog for LinkedIn jobs
  • Rate-Limited LinkedIn Fetching: Configurable rate limit for direct LinkedIn page access (default: 10 requests/min)
  • Source Filter: New dropdown to filter by source (All / LinkedIn / GitHub)
  • Toggleable Search Panels: Both GitHub and LinkedIn search can be shown/hidden via toolbar buttons

Database

  • Migration 014: Added has_easy_apply column to job_postings table

Settings

  • LinkedIn Rate Limit slider (1-60 requests per minute)

UI

  • Easy Apply indicator (bolt icon) in job table first column
  • Source filter dropdown in toolbar

@iliasaz iliasaz merged commit c10366b into main Jan 26, 2026
1 check failed
@iliasaz iliasaz deleted the feature/scrapingdog-integration branch January 26, 2026 02:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant