432 Commits

Author	SHA1	Message	Date
rafaelsideguide	5ae4d1caf5	Update single_url.ts	2024-06-04 15:28:09 -03:00
rafaelsideguide	93f3098672	build files	2024-06-04 14:54:54 -03:00
rafaelsideguide	64a4338ff0	Update single_url.ts	2024-06-04 14:40:05 -03:00
Rafael Miller	02fe470e20	Merge pull request #148 from mendableai/nsc/improvemnts-fixes-misc ... Better fallbacks for initial crawl start	2024-06-04 14:31:10 -03:00
Rafael Miller	665a40d9f4	Merge pull request #212 from mendableai/bugfix/partial-data-js-sdk ... [Bug] Improved js response and test for getting partial_data	2024-06-04 14:05:23 -03:00
rafaelsideguide	1f4c6b7a87	Update package.json	2024-06-04 13:59:48 -03:00
Rafael Miller	19c67916d4	Merge pull request #211 from mendableai/fix/rename-variables ... [Fix] Changed timeout parameter name on js sdk	2024-06-04 13:57:58 -03:00
Rafael Miller	f4f87b5374	Merge branch 'main' into bugfix/partial-data-js-sdk	2024-06-04 13:40:42 -03:00
rafaelsideguide	4e3a0495d7	updated version 0.0.12 -> 0.0.13 ... - [ ] publish	2024-06-04 12:03:55 -03:00
Rafael Miller	b80fb374e5	Merge branch 'main' into playwright-service-bug-222	2024-06-04 11:57:17 -03:00
rafaelsideguide	6920ec8a61	bugfixing. already on main	2024-06-04 11:05:50 -03:00
Nicolas	d91b725c6f	Update fly.toml	2024-06-04 00:41:15 -07:00
Nicolas	cbf8d79cce	Update pdfProcessor.ts	2024-06-04 00:13:37 -07:00
Nicolas	3fc9004ba8	Update fly.toml	2024-06-03 23:49:46 -07:00
Nicolas	2ea01f1456	Update single_url.ts	2024-06-03 23:42:39 -07:00
Nicolas	854d5b3cb3	Update single_url.ts	2024-06-03 23:32:55 -07:00
Nicolas	99059814a8	Nick:	2024-06-03 21:32:48 -07:00
Nicolas	918059ee9e	Merge branch 'main' into nsc/improvemnts-fixes-misc	2024-06-03 16:46:02 -07:00
Nicolas	38e583f66c	Update socialBlockList.test.ts	2024-06-03 16:44:23 -07:00
Nicolas	c69c89f838	Nick:	2024-06-03 16:42:42 -07:00
Nicolas	48d1ec05b2	Merge branch 'main' into nsc/improved-blocklist	2024-06-03 16:38:03 -07:00
Nicolas	d30ced4394	Merge pull request #221 from mendableai/nsc/fwd-header-auth ... feat: Ability to forward headers to reliable providers for auth etc...	2024-06-03 16:33:40 -07:00
Romain Bruyère	4987f901d1	Merge branch 'mendableai:main' into main	2024-06-03 21:29:33 +02:00
rafaelsideguide	4100cc9223	Update index.test.ts	2024-06-03 16:29:16 -03:00
rombru	3ff91ddd1f	fix: use @ instead of # for default BULL_AUTH_KEY. hash mark is reserved for URI fragments.	2024-06-03 21:28:25 +02:00
rafaelsideguide	c1aed1360e	Update index.test.ts	2024-06-03 15:51:07 -03:00
rafaelsideguide	1fc3a15149	Update single_url.ts	2024-06-03 15:24:40 -03:00
Nicolas	fde522c3e1	Update single_url.ts	2024-06-02 20:23:45 -07:00
Matt Joyce	deefe65cbe	Change the way the playwright response is parsed ... Was failing with a Type Error, but actually looked ok. This fixes the type error, and stop scraper fallback.	2024-06-01 19:16:56 +10:00
Matt Joyce	14896a9fdd	Fix PLAYWRIGHT_MICROSERVICE_URL ... It needs to end in html, otherwise scrape will 404	2024-06-01 19:03:16 +10:00
Matt Joyce	1eacad4ef3	Clarifying wait type and name	2024-06-01 18:53:03 +10:00
Matt Joyce	c516140bfb	Various Linting ... Pylint C0114: Missing module docstring C0115: Missing class docstring C0116: Missing function or method docstring C0303: Trailing whitespace Import ordering	2024-06-01 18:53:03 +10:00
Matt Joyce	2a39b5382b	Add timeout to class and provide default.	2024-06-01 18:52:42 +10:00
Nicolas	8cb62dde92	Update website_params.ts	2024-05-31 16:09:39 -07:00
Nicolas	3b8059edb6	Update single_url.ts	2024-05-31 15:43:06 -07:00
Nicolas	6bea803120	Nick:	2024-05-31 15:39:54 -07:00
Nicolas	2139129296	Nick: v12	2024-05-31 11:39:55 -07:00
Nicolas	260e31c68b	Merge branch 'nsc/new-pricing'	2024-05-30 16:08:31 -07:00
Nicolas	aa8133ca7f	Update load-testing-example.ts	2024-05-30 16:07:14 -07:00
Nicolas	0c115c6181	Merge pull request #216 from mendableai/nsc/new-pricing ... feat: New pricing/limits changes	2024-05-30 15:36:59 -07:00
Nicolas	6860ace4af	Nick:	2024-05-30 15:07:49 -07:00
Nicolas	6ceb7ff50a	Nick:	2024-05-30 14:46:55 -07:00
Nicolas	33f10a7f91	Nick: fixes	2024-05-30 14:42:32 -07:00
Nicolas	ace46f340b	Nick: new limits, new pricing	2024-05-30 14:31:36 -07:00
rafaelsideguide	2b763d848b	improved js response and test for getting partial_data	2024-05-30 08:44:38 -03:00
rafaelsideguide	5b8b6902e7	Update index.ts	2024-05-30 08:25:13 -03:00
Nicolas	6c939d534d	Nick: small refactor	2024-05-29 19:43:51 -07:00
Eric Ciarla	37915e11e8	Final push	2024-05-29 21:18:24 -04:00
Eric Ciarla	a0e404f94e	init commit	2024-05-29 18:56:57 -04:00
rafaelsideguide	ee9a2184e2	Added custom scraping conditions for readme docs	2024-05-29 13:39:43 -03:00