388 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Nicolas	098db17913	Update index.ts	2024-05-15 17:37:09 -07:00
Nicolas	93b1f0334e	Update index.test.ts	2024-05-15 17:35:06 -07:00
Nicolas	123fb784ca	Update index.test.ts	2024-05-15 17:29:22 -07:00
Nicolas	4a6cfb6097	Update index.test.ts	2024-05-15 17:22:29 -07:00
Nicolas	6ca368327f	Merge branch 'main' into test/crawl-options	2024-05-15 17:18:25 -07:00
Nicolas	24be4866c5	Nick:	2024-05-15 17:16:20 -07:00
Nicolas	ade4e05cff	Nick: working	2024-05-15 17:13:04 -07:00
Nicolas	bfccaf670d	Nick: fixes most of it	2024-05-15 15:30:37 -07:00
rafaelsideguide	d91043376c	not working yet	2024-05-15 18:54:40 -03:00
rafaelsideguide	fa014defc7	Fixing child links only bug	2024-05-15 18:35:09 -03:00
Nicolas	2ba743fb1a	Merge pull request #27 from eltociear/patch-1 ... refactor: fix typo in WebScraper/index.ts	2024-05-15 13:28:38 -07:00
Nicolas	0663d78324	Merge pull request #119 from chand1012/main ... Add Docker Compose for easy self hosting	2024-05-15 13:27:40 -07:00
Nicolas	eb36d4b3bd	Update SELF_HOST.md	2024-05-15 13:25:39 -07:00
rafaelsideguide	da8d94105d	fixed for testing the crawl algorithm only	2024-05-15 17:16:03 -03:00
Nicolas	95ffaa2236	Update crawl.test.ts	2024-05-15 12:58:02 -07:00
Nicolas	f15b8f855e	Update crawl.json	2024-05-15 12:57:24 -07:00
Nicolas	98dd672d0a	Update crawl.json	2024-05-15 12:55:04 -07:00
Nicolas	499671c87f	Update crawl.test.ts	2024-05-15 12:50:13 -07:00
Nicolas	58053eb423	Update rate-limiter.ts	2024-05-15 12:47:35 -07:00
Nicolas	4745d114be	Update crawl.test.ts	2024-05-15 12:42:14 -07:00
Nicolas	1601e93d69	Merge branch 'main' into test/crawl-options	2024-05-15 12:34:47 -07:00
Nicolas	3678d3c986	Merge branch 'main' of https://github.com/mendableai/firecrawl	2024-05-15 12:11:18 -07:00
Nicolas	fd82982a31	Nick:	2024-05-15 12:11:16 -07:00
rafaelsideguide	4925ee59f6	added crawl test suite	2024-05-15 15:50:50 -03:00
Nicolas	ed211dc7f8	Merge pull request #149 from mendableai/nsc/speed-up-crawl-4x ... feat: 4x-5x faster crawler (fast mode)	2024-05-15 11:48:32 -07:00
Nicolas	1b0d6341d3	Update index.ts	2024-05-15 11:48:12 -07:00
Nicolas	d10f81e7fe	Nick: fixes	2024-05-15 11:28:20 -07:00
Nicolas	87570bdfa1	Update index.ts	2024-05-15 11:06:03 -07:00
Ikko Eltociear Ashimine	e91c122c69	Merge branch 'main' into patch-1	2024-05-15 12:14:52 +09:00
Nicolas	27e1e22a0a	Update index.test.ts	2024-05-14 12:28:25 -07:00
Nicolas	a0fdc6f7c6	Nick:	2024-05-14 12:12:40 -07:00
Nicolas	7f31959be7	Nick:	2024-05-14 12:04:36 -07:00
Nicolas	8a72cf556b	Nick:	2024-05-13 21:10:58 -07:00
Nicolas	a96fc5b96d	Nick: 4x speed	2024-05-13 20:45:11 -07:00
Nicolas	e26008a833	Merge branch 'main' of https://github.com/mendableai/firecrawl	2024-05-13 19:54:13 -07:00
Nicolas	512449e1aa	Nick: v21	2024-05-13 19:54:12 -07:00
Nicolas	bd27b0e17e	Merge pull request #142 from mendableai/doc/crawl-limit-default ... [Doc] Added default value for crawlOptions.limit	2024-05-13 18:38:09 -07:00
Nicolas	aa0c8188c9	Nick: 408 handling	2024-05-13 18:34:00 -07:00
Nicolas	999176d576	Merge branch 'main' of https://github.com/mendableai/firecrawl	2024-05-13 13:57:34 -07:00
Nicolas	f3ec21d9c4	Update runWebScraper.ts	2024-05-13 13:57:22 -07:00
Nicolas	c9133f3d15	Merge pull request #145 from mendableai/nsc/timeout-scrape ... Timeout on /scrape	2024-05-13 13:07:25 -07:00
Nicolas	65d89afba9	Nick:	2024-05-13 13:01:43 -07:00
Nicolas	3f090ffd7c	Merge pull request #144 from mendableai/feat/gpt-4o ... Update models.ts	2024-05-13 12:24:30 -07:00
Eric Ciarla	4cc46d4af8	Update models.ts	2024-05-13 15:23:31 -04:00
Nicolas	2ce045912f	Nick: disable vision right now	2024-05-13 10:56:08 -07:00
rafaelsideguide	4737fe8711	Added missing instruction	2024-05-13 13:47:49 -03:00
chand1012	5cbce060ed	chore: Update docker-compose.yaml with default values for PORT and HOST	2024-05-10 17:26:00 -04:00
chand1012	b498e9881c	chore: Update docker-compose.yaml network configuration	2024-05-10 17:23:22 -04:00
chand1012	2021a822ff	chore: Add firecrawl network to docker-compose.yaml	2024-05-10 17:20:33 -04:00
chand1012	0245066009	chore: Update docker-compose.yaml with default values for REDIS_URL and PLAYWRIGHT_MICROSERVICE_URL	2024-05-10 17:15:32 -04:00