Commit Graph

1546 Commits

Author SHA1 Message Date
Gergő Móricz
9579f03c4b fix: import resolution 2024-08-21 20:16:06 +02:00
Gergő Móricz
6104d74213 fix(sentry): drop profiling sample rate 2024-08-21 20:12:47 +02:00
Gergő Móricz
3d5dc9d90a feat(sentry): add log + server name 2024-08-21 19:39:10 +02:00
Nicolas
79f5d49d3f
Merge pull request #562 from mendableai/nsc/sentry
Added Sentry Monitoring
2024-08-21 14:22:29 -03:00
Gergő Móricz
85ff0c311e Add worker ID to job attribute 2024-08-21 19:21:29 +02:00
Gergő Móricz
3ad9bf7ac0 Update GH Actions deployment 2024-08-21 19:15:25 +02:00
Gergő Móricz
920702cdde Update builder to handle uploading sourcemaps 2024-08-21 19:08:03 +02:00
Gergő Móricz
86942728e3 Add metadata for queue-worker and Express 2024-08-21 17:58:27 +02:00
Nicolas
35decb1af2 Nick: 2024-08-21 12:35:03 -03:00
rafaelsideguide
af0e47a30e Merge remote-tracking branch 'origin/v1/node-sdk' into v1/python-sdk 2024-08-21 12:09:53 -03:00
rafaelsideguide
52abec41c2 fixing delete 2024-08-21 10:35:50 -03:00
Nicolas
db8c84ff0f Update requests.http 2024-08-21 10:19:37 -03:00
rafaelsideguide
b66553867e reverting delete, fixed express bug on checkCredits 2024-08-21 09:28:20 -03:00
rafaelsideguide
138437d616 commenting out delete, crashing on fire-engine
Some checks are pending
Fly Deploy / Pre-deploy checks (push) Waiting to run
Fly Deploy / Test Suite (push) Blocked by required conditions
Fly Deploy / Python SDK Tests (push) Blocked by required conditions
Fly Deploy / JavaScript SDK Tests (push) Blocked by required conditions
Fly Deploy / Go SDK Tests (push) Blocked by required conditions
Fly Deploy / Deploy app (push) Blocked by required conditions
Fly Deploy / Build and publish Python SDK (push) Blocked by required conditions
Fly Deploy / Build and publish JavaScript SDK (push) Blocked by required conditions
2024-08-21 08:11:24 -03:00
rafaelsideguide
5e48bec1fd commenting out delete, crashing on fire-engine 2024-08-21 08:10:46 -03:00
Nicolas
90b32f16c8 Nick: fixes 2024-08-20 21:38:11 -03:00
Nicolas
819ad50af3 Update fireEngine.ts 2024-08-20 21:16:33 -03:00
rafaelsideguide
e9d6ca197e tests passing now 2024-08-20 20:00:41 -03:00
Nicolas
1b3ad60a2c Reapply "Merge pull request #561 from mendableai/bug/dealing-with-dns-error"
This reverts commit ffe11a5bf7.
2024-08-20 19:22:09 -03:00
Nicolas
441628998f Reapply "Merge pull request #561 from mendableai/bug/dealing-with-dns-error"
This reverts commit ffe11a5bf7.
2024-08-20 19:16:48 -03:00
Nicolas
ffe11a5bf7 Revert "Merge pull request #561 from mendableai/bug/dealing-with-dns-error"
Some checks are pending
Fly Deploy / Pre-deploy checks (push) Waiting to run
Fly Deploy / Test Suite (push) Blocked by required conditions
Fly Deploy / Python SDK Tests (push) Blocked by required conditions
Fly Deploy / JavaScript SDK Tests (push) Blocked by required conditions
Fly Deploy / Go SDK Tests (push) Blocked by required conditions
Fly Deploy / Deploy app (push) Blocked by required conditions
Fly Deploy / Build and publish Python SDK (push) Blocked by required conditions
Fly Deploy / Build and publish JavaScript SDK (push) Blocked by required conditions
This reverts commit 2030ec6031, reversing
changes made to f494d2b707.
2024-08-20 18:16:11 -03:00
Gergő Móricz
5818236659 fix: remove rawHtml properly 2024-08-20 22:51:12 +02:00
rafaelsideguide
0f48823c9c Merge remote-tracking branch 'origin/v1-webscraper' into v1/node-sdk 2024-08-20 17:24:59 -03:00
rafaelsideguide
9d64c8eedc screenshot should not be undefined also 2024-08-20 17:24:45 -03:00
Gergő Móricz
1368f9a87f fix: treat existing screenshot as a scraper success condition 2024-08-20 22:24:18 +02:00
rafaelsideguide
70d81ca69e Merge remote-tracking branch 'origin/v1-webscraper' into v1/node-sdk 2024-08-20 17:08:07 -03:00
rafaelsideguide
537fc689b6 fixing request 2024-08-20 17:08:02 -03:00
Nicolas
2030ec6031
Merge pull request #561 from mendableai/bug/dealing-with-dns-error
[Bug] Added a way for dealing with DNS without IP resolution
2024-08-20 16:59:09 -03:00
rafaelsideguide
f98be7d94e Update fireEngine.ts 2024-08-20 16:53:01 -03:00
Nicolas
0c48c8a436 Nick: billing for map 2024-08-20 16:43:46 -03:00
Gergő Móricz
f494d2b707 prioritize search lower 2024-08-20 21:18:31 +02:00
Nicolas
59eb552df5
Merge pull request #560 from mendableai/feat/cancel-fire-engine
[Feat] Added check job and cancel to fire-engine requests
2024-08-20 15:43:31 -03:00
rafaelsideguide
1f27182a13 added try catch 2024-08-20 15:42:39 -03:00
Gergő Móricz
efb91f9ca5 fix(search): delete jobs after done 2024-08-20 20:29:08 +02:00
Nicolas
39388cdc35 Update crawl.ts 2024-08-20 14:41:43 -03:00
Nicolas
674adee144 Merge branch 'v1-webscraper' of https://github.com/mendableai/firecrawl into v1-webscraper 2024-08-20 14:41:05 -03:00
Nicolas
b36faeaf54 Nick: 2024-08-20 14:39:52 -03:00
Nicolas
2b221fd4a9
Merge pull request #559 from mendableai/nsc/Nick-set-the-crawl-limit-to-the-remaining-credits
Set the crawl limit to the remaining credits
2024-08-20 14:38:10 -03:00
Gergő Móricz
cf32893c2e add strict enforcement + move crawlerOptions to top-level in /crawl 2024-08-20 19:31:26 +02:00
rafaelsideguide
e326249a57 added check job and cancel to fire-engine requests 2024-08-20 14:26:42 -03:00
Gergő Móricz
70d50b3640 fix(queue-worker): move dotenv config up 2024-08-20 19:25:19 +02:00
Nicolas
c5ad4dedeb Update crawl.ts 2024-08-20 14:19:20 -03:00
Nicolas
de0dc20a02 Update credit_billing.ts 2024-08-20 14:18:14 -03:00
Nicolas
5abd26a267 Nick: set the crawl limit to the remaining credits 2024-08-20 14:16:54 -03:00
Nicolas
e200ec9e12 Nick: 2024-08-20 12:24:14 -03:00
Nicolas
55dad82df1 Nick: fixed map search 2024-08-20 12:17:53 -03:00
Nicolas
27903247b6 Nick: map tests and fixes 2024-08-20 12:04:08 -03:00
rafaelsideguide
fa89d2e535 v1 support for crawl/monitor status 2024-08-20 10:37:24 -03:00
rafaelsideguide
7727302ef1 Merge remote-tracking branch 'origin/v1-webscraper' into v1/node-sdk 2024-08-20 09:22:55 -03:00
rafaelsideguide
e160d5529a fixed test 2024-08-20 09:22:38 -03:00