937 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Nicolas	c08db830df	Merge pull request #268 from mendableai/nsc/abs-path-fix ... Fixed bugs associated with absolute path replacements	2024-06-11 13:20:42 -07:00
Nicolas	1e3e06a1d5	Update replacePaths.test.ts	2024-06-11 13:02:39 -07:00
Nicolas	2239e03269	Update replacePaths.test.ts	2024-06-11 12:54:02 -07:00
Nicolas	520739c9f4	Nick: fixed bugs associated with absolute path replacements	2024-06-11 12:43:16 -07:00
Nicolas	788abdce6e	Merge branch 'main' of https://github.com/mendableai/firecrawl	2024-06-11 12:08:51 -07:00
Nicolas	b87725c683	Update openapi.json	2024-06-11 12:08:49 -07:00
rafaelsideguide	ee282c3d55	Added allowBackwardCrawling option	2024-06-11 15:24:39 -03:00
rafaelsideguide	a9f93c2f1e	Added route to clean completed jobs and a github action cron that triggers every 24h	2024-06-11 14:18:05 -03:00
Eric Ciarla	06b0d01fd4	Update examples	2024-06-11 12:23:36 -04:00
Eric Ciarla	00c23855b1	Update examples	2024-06-11 11:46:35 -04:00
Nicolas	da38dad9a7	Merge branch 'main' of https://github.com/mendableai/firecrawl	2024-06-10 18:26:31 -07:00
Nicolas	9390816c1b	Update openapi.json	2024-06-10 18:26:25 -07:00
Nicolas	15e791ffb1	Merge pull request #263 from mendableai/nsc/pageoptions-crawler ... ignoreSitemap feature, pageOptions now respected in the initial crawl as well	2024-06-10 18:22:51 -07:00
Nicolas	f6b06ac27a	Nick: ignoreSitemap, better crawling algo	2024-06-10 18:12:41 -07:00
Nicolas	1bd0327e1a	Merge branch 'main' into nsc/pageoptions-crawler	2024-06-10 17:15:10 -07:00
Nicolas	149d79a529	Merge pull request #260 from mendableai/nsc/fix-deadlocks ... Moving from fetch to axios and preventing deadlocks by setting timeouts on fallbacks	2024-06-10 17:06:33 -07:00
Nicolas	99f2ffd6d5	Update webhook.ts	2024-06-10 17:03:10 -07:00
Nicolas	7ae9778642	Update single_url.ts	2024-06-10 16:57:31 -07:00
Nicolas	913c1dd568	Nick: fetch -> axios and fix timeouts	2024-06-10 16:49:03 -07:00
Nicolas	3091f0134c	Nick:	2024-06-10 16:27:10 -07:00
Matt Joyce	827354a116	Added logging to python sdk FIRECRAWL_LOGGING_LEVEL ... Instantiates the logger early and depends on env to set.	2024-06-10 21:21:23 +10:00
Nicolas	aafd23fa8a	Merge pull request #252 from mattjoyce/fix-208-py-sdk-interval-poll-name ... Fix 208 py sdk interval poll name	2024-06-08 21:33:17 -07:00
Matt Joyce	6fd9ce1c89	type hints and linting	2024-06-08 11:46:52 +10:00
Matt Joyce	7477c5e5bd	Use error handler consistently	2024-06-08 11:28:51 +10:00
Matt Joyce	9f306736af	More detailed error handling	2024-06-08 11:18:30 +10:00
Matt Joyce	c71ea7a795	Prepare headers consistently	2024-06-08 11:08:26 +10:00
Matt Joyce	8f9a165c2f	Lint - whitespace	2024-06-08 08:03:02 +10:00
Matt Joyce	5f0df596ec	Align param name with JS SDK ... timeout becomes poll_interval	2024-06-08 07:37:08 +10:00
Nicolas	f24ca76618	Nick: removing rate limit emails for now	2024-06-07 10:39:11 -07:00
rafaelsideguide	556c57648e	Update fly.yml	2024-06-07 09:40:40 -03:00
rafaelsideguide	0e89f8b9a3	fixing workflow	2024-06-07 09:35:56 -03:00
Nicolas	98d82c4cec	Update search.ts	2024-06-06 20:02:21 -07:00
Nicolas	5e80f8af87	Nick: llm extract 50	2024-06-06 18:35:44 -07:00
rafaelsideguide	7b7a6f8a39	Merge branch 'main' of https://github.com/mendableai/firecrawl	2024-06-06 17:51:28 -03:00
rafaelsideguide	f2695df215	updated sdk versions	2024-06-06 17:51:12 -03:00
Rafael Miller	8dabf68972	Merge pull request #250 from mendableai/194-fixing-2 ... fixing minor problems on workflow	2024-06-06 17:37:22 -03:00
rafaelsideguide	560f256a35	fixing minor problems on workflow	2024-06-06 17:36:48 -03:00
Rafael Miller	8bfec9e8de	Merge pull request #248 from mendableai/194-sdk-ci-pipeline-for-publishing-pythonnode-sdk ... 194 sdk ci pipeline for publishing pythonnode sdk	2024-06-06 17:28:51 -03:00
rafaelsideguide	e09580444f	Merge branch '194-sdk-ci-pipeline-for-publishing-pythonnode-sdk' of https://github.com/mendableai/firecrawl into 194-sdk-ci-pipeline-for-publishing-pythonnode-sdk	2024-06-06 17:23:54 -03:00
rafaelsideguide	59e8615050	Update fly.yml	2024-06-06 17:23:45 -03:00
rafaelsideguide	9d0643edbf	Merge branch 'main' of https://github.com/mendableai/firecrawl into 194-sdk-ci-pipeline-for-publishing-pythonnode-sdk	2024-06-06 17:22:23 -03:00
rafaelsideguide	34742e8bcf	Update fly.yml	2024-06-06 17:00:28 -03:00
Rafael Miller	4c3bfe4eb5	Merge pull request #246 from mendableai/194-sdk-ci-pipeline-for-publishing-pythonnode-sdk ... [Feat] CI/CD for publishing js and python SDKs	2024-06-06 16:53:42 -03:00
rafaelsideguide	f5318ea7d7	Update index.test.ts	2024-06-06 16:50:20 -03:00
rafaelsideguide	cd7f9abcec	Update index.test.ts	2024-06-06 16:44:46 -03:00
rafaelsideguide	7b9b668b95	Update index.test.ts	2024-06-06 16:36:51 -03:00
rafaelsideguide	82e0ed4cd3	Update index.test.ts	2024-06-06 16:33:27 -03:00
rafaelsideguide	dac7612be2	Merge branch 'main' of https://github.com/mendableai/firecrawl into 194-sdk-ci-pipeline-for-publishing-pythonnode-sdk	2024-06-06 16:07:25 -03:00
Nicolas	c2ad358390	Nick:	2024-06-06 12:05:20 -07:00
rafaelsideguide	79ec9f04dc	Merge branch 'main' of https://github.com/mendableai/firecrawl into 194-sdk-ci-pipeline-for-publishing-pythonnode-sdk	2024-06-06 15:58:14 -03:00