Commit Graph

525 Commits

Author SHA1 Message Date
Nicolas
9b5e5b8722
Merge pull request #183 from mendableai/test-sdks
[Test] Added sdk e2e tests
2024-05-28 11:32:29 -07:00
Nicolas
41b98f15a8
Update README.md 2024-05-27 22:57:13 -07:00
rafaelsideguide
71187b03a2 added timeout 2024-05-27 16:48:08 -03:00
rafaelsideguide
d5c83803cd fixing idempotency test 2024-05-27 16:35:01 -03:00
rafaelsideguide
41c4ef6a82 dotenv was missing 2024-05-27 16:23:57 -03:00
rafaelsideguide
6b58da1c96 jest 2024-05-27 16:01:48 -03:00
rafaelsideguide
e87d39e6ec typo 2024-05-27 15:55:48 -03:00
rafaelsideguide
127d2db1dd added js/ts sdk tests 2024-05-27 15:54:09 -03:00
rafaelsideguide
d0c4b24a0e missing redis 2024-05-27 14:36:32 -03:00
rafaelsideguide
952ccd8755 envs 2024-05-27 14:34:39 -03:00
rafaelsideguide
f32c16258a missing node setup 2024-05-27 14:32:34 -03:00
rafaelsideguide
a9b68d95d8 Update test.py 2024-05-27 14:28:44 -03:00
rafaelsideguide
c410dbe5bd Update python-tests.yml 2024-05-27 14:24:30 -03:00
rafaelsideguide
667d3e4c4f Merge branch 'test-sdks' of https://github.com/mendableai/firecrawl into test-sdks 2024-05-27 14:23:39 -03:00
rafaelsideguide
19decd1062 fixing workflow 2024-05-27 14:21:33 -03:00
Rafael Miller
3c8edf683c
Merge branch 'main' into test-sdks 2024-05-27 14:15:18 -03:00
rafaelsideguide
63772ea711 added github action workflow 2024-05-27 14:14:00 -03:00
Nicolas
1ef307cb6f Nick: checks 2024-05-27 10:01:12 -07:00
Nicolas
a9e45cdb15
Merge pull request #191 from mattjoyce/main
Various PyPi Metadata
2024-05-26 18:15:19 -07:00
Nicolas
1de53cc4d0 Nick: fixes 2024-05-26 18:15:05 -07:00
Nicolas
efb821d63b
Merge branch 'main' into main 2024-05-26 18:12:23 -07:00
Nicolas
ed4226fd1f
Update setup.py 2024-05-26 18:11:54 -07:00
Nicolas
1bbfb98d7e
Merge pull request #186 from Keredu/main
Limit on /search is not deterministic
2024-05-26 18:08:16 -07:00
Nicolas
67a53a9ae0
Merge pull request #190 from simonha9/simonha9/improve-rate-limit-error-msg
Feat: Provide more details for 429 error msg
2024-05-26 18:07:42 -07:00
Nicolas
7e2df7bd5e Update auth.ts 2024-05-26 18:07:21 -07:00
Nicolas
7948c6cee2 Nick: fixed pip issues 2024-05-26 18:03:37 -07:00
Matt Joyce
b061e12030 added python versions requirement
this is inline with requests module, a critical dependency
2024-05-26 11:37:47 +10:00
Matt Joyce
f00dffbbb1 added misc PyPi keys
help potential users find and understand the purpose and status of the project.
2024-05-26 11:36:29 +10:00
Matt Joyce
cd7f260288 Added PyPi classifiers
These classifiers will help potential users find and understand the purpose and status of the project.  use python 3.8 as the base, because that's what module 'requests' needs.
2024-05-26 11:33:28 +10:00
Matt Joyce
e5c6ac23fe Added long description to PyPi
https://packaging.python.org/en/latest/guides/making-a-pypi-friendly-readme/
2024-05-26 10:01:35 +10:00
Simon H
115204e6b6 Feat: Provide more details for 429 error msg
- Added better error code for when rate limit exceeded including
consumed/remaining points, reset date and retry-after seconds
2024-05-25 12:03:20 -04:00
Keredu
2192978f91 Limit on /search is not deterministic 2024-05-25 00:12:26 +02:00
Nicolas
e5c8719554 Update blocklist.ts 2024-05-24 14:53:04 -07:00
rafaelsideguide
397769c7e3 added python sdk e2e tests with pytest
some of them are still missing though
2024-05-24 17:56:27 -03:00
Rafael Miller
4ce28593b2
Merge pull request #132 from mendableai/feat/idempotency-key
[Feat] Added idempotency key to crawl route
2024-05-24 14:22:50 -03:00
rafaelsideguide
d39860c08b Merge branch 'main' into feat/idempotency-key 2024-05-24 14:15:37 -03:00
Nicolas
605ba4c031
Merge pull request #178 from mattjoyce/main
Fix FIRECRAWL_API_URL bug, also various PyLint fixes
2024-05-24 09:49:42 -07:00
Nicolas
8c380d70a5
Update firecrawl.py 2024-05-24 09:48:48 -07:00
Nicolas
65fe9c4f80
Merge branch 'main' into main 2024-05-24 09:47:12 -07:00
Rafael Miller
53a7ec0f6e
Removed hard coded timeout 2024-05-24 13:46:16 -03:00
Nicolas
e0d979edad
Merge pull request #176 from mendableai/bug/data-check-in-python-sdk
[Bug] Added data check for python SDK
2024-05-24 09:45:39 -07:00
Nicolas
53a214cefb
Merge pull request #168 from mendableai/nsc/allowed-keywords-in-blocklist
feat: Allow privacy/legal/ other pages in social media websites
2024-05-24 09:43:15 -07:00
Nicolas
e166c07690
Merge pull request #170 from qyou/fix-hardcode-timeout
update: wait until body attached in playwright-service
2024-05-24 09:41:27 -07:00
Rafael Miller
c00580eabd
Merge pull request #181 from JakobStadlhuber/feat/proxy-support
[Feat] Added proxy and media blocking support for Playwright
2024-05-24 13:34:34 -03:00
Jakob Stadlhuber
9fc5a0ff98 Update comment in .env.example for proxy settings
This commit modifies the comment in .env.example to specify that proxy settings are for Playwright. This clarification aims to provide users a more clear context about when and why these proxy settings are used.
2024-05-24 17:45:59 +02:00
Jakob Stadlhuber
b001aded46 Add proxy and media blocking configurations
Updated environment variables and application settings to include proxy configurations and media blocking option. The proxy settings allow users to use a proxy service, while the media blocking is an optional feature that can help save bandwidth. Changes have been made in the .env.example, docker-compose.yaml, and main.py files.
2024-05-24 17:41:34 +02:00
Rafael Miller
6a5b9ca314
Merge pull request #180 from mendableai/added-issue-templates
Update issue templates
2024-05-24 10:17:40 -03:00
Rafael Miller
9562c837eb
Update issue templates 2024-05-24 09:34:43 -03:00
rafaelsideguide
c201ea1986 added idempotency key to python sdk 2024-05-23 12:52:59 -03:00
rafaelsideguide
35927a65a5 Merge branch 'main' into feat/idempotency-key 2024-05-23 12:20:06 -03:00