Thin adapter that mimics a subset of the Firecrawl API.
Bob
43bb5955c1
feat: initial POC with NestJS, hexagonal architecture, vitest tests, docker support
|
1 ماه پیش | |
|---|---|---|
| src | 1 ماه پیش | |
| test | 1 ماه پیش | |
| .dockerignore | 1 ماه پیش | |
| CHANGELOG.md | 1 ماه پیش | |
| Dockerfile | 1 ماه پیش | |
| README.md | 1 ماه پیش | |
| docker-compose.yaml | 1 ماه پیش | |
| nest-cli.json | 1 ماه پیش | |
| package.json | 1 ماه پیش | |
| tsconfig.json | 1 ماه پیش | |
| vitest.config.ts | 1 ماه پیش |
README.md
Firecrawl Browserless Adapter (POC)
Thin adapter that mimics a subset of the Firecrawl API using:
- Direct HTTP fetch for SSR/static pages.
- Browserless (headless Chromium) as fallback / SPA renderer.
- Readability + Turndown to extract clean Markdown.
Tech stack:
- NestJS, TypeScript, hexagonal architecture.
- Vitest for tests.
- Docker + docker-compose only; no host dependencies required.
Endpoints:
- GET /v1/health -> { success: true, status: "ok", service: "firecrawl-browserless-adapter" }
- POST /v1/scrape
- Body: { url: string, render?: boolean }
- Response: { url, title, markdown, html?, metadata: { source } }
Usage (local):
- docker compose up --build
- curl http://localhost:3000/v1/health
- curl -X POST http://localhost:3000/v1/scrape -H "Content-Type: application/json" -d '{"url": "https://example.com"}'
SPA / render mode:
- Set render=true to force Browserless for SPAs.
- Without it, the adapter tries direct fetch first and falls back if content looks incomplete.
Notes:
- This is a POC; not production-hardened.
- No auth, no rate limiting, basic error handling only.