Nicolas
968a2dc753
Nick:
2024-07-29 18:37:09 -04:00
Nicolas
04942bb9de
Nick:
2024-07-29 18:31:43 -04:00
Nicolas
267d4681bf
Merge branch 'main' of https://github.com/mendableai/firecrawl
2024-07-29 17:21:15 -04:00
Nicolas
b4833c1694
Nick: increasing default timeout to 45s
2024-07-29 17:21:11 -04:00
Nicolas
7fa08100bf
Merge pull request #414 from NiuBlibing/support_model_name
...
Fly Deploy / Pre-deploy checks (push) Waiting to run
Fly Deploy / Test Suite (push) Blocked by required conditions
Fly Deploy / Python SDK Tests (push) Blocked by required conditions
Fly Deploy / JavaScript SDK Tests (push) Blocked by required conditions
Fly Deploy / Deploy app (push) Blocked by required conditions
Fly Deploy / Build and publish Python SDK (push) Blocked by required conditions
Fly Deploy / Build and publish JavaScript SDK (push) Blocked by required conditions
support custom models
2024-07-29 13:21:29 -04:00
Nicolas
4c9d62f6d3
Nick: fixing sitemap fallback
2024-07-26 18:25:44 -04:00
Nicolas
091924a636
Nick: moving machines from mia to virginia
2024-07-26 17:37:46 -04:00
Nicolas
cb97871ff9
Merge branch 'main' of https://github.com/mendableai/firecrawl
2024-07-26 17:21:11 -04:00
Nicolas
ff4266f09e
Update pdfProcessor.ts
2024-07-26 17:21:09 -04:00
Nicolas
0c2e3a72cc
Merge pull request #460 from mendableai/nsc/admin-router
...
Admin router + Improve redis notifications
2024-07-26 12:16:14 -04:00
rafaelsideguide
96cec2a673
fix checking scrape log success content length
2024-07-26 12:00:52 -03:00
Nicolas
542270f4c2
Merge pull request #461 from mendableai/nsc/small-handle-for-client-side-errors
...
Client side error handling
2024-07-25 20:53:10 -04:00
Nicolas
dc6f825270
Update email_notification.ts
2024-07-25 20:43:50 -04:00
Nicolas
f82ca3be17
Nick:
2024-07-25 19:53:29 -04:00
Nicolas
01fab6e036
Update single_url.ts
2024-07-25 17:51:41 -04:00
Nicolas
56042d090c
Update single_url.ts
2024-07-25 17:48:44 -04:00
Nicolas
88f5efce8f
Merge branch 'feat/scrape-monitoring'
2024-07-25 17:44:21 -04:00
Nicolas
3242872503
Update single_url.ts
2024-07-25 17:43:55 -04:00
Nicolas
ffd430f198
Merge pull request #457 from JakobStadlhuber/Readiness-Liveness-Probes
...
Readiness liveness probes
2024-07-25 17:20:31 -04:00
Nicolas
7129d7993e
Update v0.ts
2024-07-25 17:19:45 -04:00
rafaelsideguide
e0954d7f59
Merge branch 'main' of https://github.com/mendableai/firecrawl
2024-07-25 17:48:43 -03:00
rafaelsideguide
81aa919262
fix
2024-07-25 17:47:43 -03:00
Nicolas
10e80f00cf
Merge branch 'main' into nsc/admin-router
2024-07-25 16:46:38 -04:00
Nicolas
11e6b2680e
Merge pull request #455 from mendableai/feat/scrape-monitoring
...
Add scrape monitoring
2024-07-25 16:27:07 -04:00
Nicolas
e5b797549e
Merge branch 'main' into feat/scrape-monitoring
2024-07-25 16:21:02 -04:00
Nicolas
50d2426fc4
Update scrape-events.ts
2024-07-25 16:20:29 -04:00
Nicolas
28a8a98491
Update admin.ts
2024-07-25 14:58:14 -04:00
Nicolas
2014d9dd2e
Nick: admin router
2024-07-25 14:54:20 -04:00
Nicolas
a75d6889c7
Merge pull request #450 from mendableai/feat/logger
...
[wip] Added logger
2024-07-25 14:40:19 -04:00
rafaelsideguide
1f1c068eea
changing from error to debug
2024-07-25 10:00:50 -03:00
rafaelsideguide
e720e1bacf
Merge remote-tracking branch 'origin/main' into feat/logger
2024-07-25 09:49:27 -03:00
rafaelsideguide
309728a482
updated logs
2024-07-25 09:48:06 -03:00
Nicolas
2c1221750b
Merge pull request #449 from mendableai/bugfix/malformed-url-sitemap
...
Added regex for links in sitemap
2024-07-24 20:37:35 -04:00
Nicolas
6ad7e24403
Update ingestion.tsx
2024-07-24 18:15:51 -04:00
Nicolas
92843a356d
Merge branch 'main' of https://github.com/mendableai/firecrawl
2024-07-24 18:13:36 -04:00
Nicolas
1e13ddbe8e
Nick: changes to the ui component
2024-07-24 18:13:34 -04:00
Gergő Móricz
623b547292
fix(fly.toml): scale up memory limit
2024-07-24 23:39:00 +02:00
Nicolas
15890772be
Scale bump
2024-07-24 16:56:19 -04:00
Eric Ciarla
a4bccbe3bb
Firecrawl UI Template
...
Firecrawl UI template
2024-07-24 15:05:55 -04:00
Eric Ciarla
a62c0730c1
Delete package-lock.json
2024-07-24 15:00:19 -04:00
Eric Ciarla
4cb091ad05
Update .gitignore
2024-07-24 14:59:34 -04:00
Eric Ciarla
4596d0b2e6
Add ReadMe and LICENSE
2024-07-24 14:56:53 -04:00
Eric Ciarla
9654721bf2
Vite commit
2024-07-24 14:27:50 -04:00
rafaelsideguide
cc98f83fda
added failed and completed log events
2024-07-24 15:25:36 -03:00
Jakob Stadlhuber
2dc7be3869
Remove liveness and readiness probes from worker.yaml
...
This commit removes the liveness and readiness probes configuration from the Kubernetes worker manifest. Additionally, a Service definition for the worker application has been removed. These changes might be necessary to update the deployment strategy or simplify the configuration.
2024-07-24 19:38:54 +02:00
Jakob Stadlhuber
d68f349109
Update Kubernetes YAMLs and add worker service
...
Refactored container configurations in worker, api, and playwright-service YAMLs to streamline syntax and add missing fields. Added a service definition for the worker component and included a new environment variable in the configmap for rate-limiting. These changes enhance configuration clarity and ensure proper resource definitions.
2024-07-24 19:31:37 +02:00
Jakob Stadlhuber
f26bda2477
Update Docker build paths in Kubernetes setup README
...
Corrected relative paths for Docker build commands to ensure the appropriate directories are targeted. This fix is crucial for successful image builds and deployment consistency in the Kubernetes cluster setup.
2024-07-24 19:06:19 +02:00
Jakob Stadlhuber
895e80caa4
Add liveness and readiness probes to Kubernetes configs
...
Introduced liveness and readiness probes for the Playwright service, API, and worker components. This ensures that Kubernetes can better manage the health and availability of these services by periodically checking their endpoints. This enhancement will improve the robustness and reliability of the deployed applications.
2024-07-24 19:00:23 +02:00
Jakob Stadlhuber
be9e7f9edf
Update Kubernetes configs for playwright-service, api, and worker
...
Added new ConfigMap for playwright-service and adjusted existing references.
Applied imagePullPolicy: Always to ensure all images are updated promptly.
Updated README to include --no-cache for Docker build instructions.
2024-07-24 18:54:16 +02:00
Gergo Moricz
60c74357df
feat(ScrapeEvents): log queue events
2024-07-24 18:44:14 +02:00
