Commit Graph

258 Commits

Author SHA1 Message Date
Yanlong Wang
f171e54ac9
fix: log charge amount 2024-05-14 17:25:59 +08:00
Han Xiao
04eb6b3087
Update README.md 2024-05-13 14:32:01 +02:00
yanlong.wang
ffc6899acd
chore: reduce resource 2024-05-13 18:35:11 +08:00
yanlong.wang
e417cd8a53
fix: tidyMarkdown feature in turndown rues 2024-05-09 15:15:15 +08:00
Yanlong Wang
36bf5d96b5
fix: remove tidyMarkdown at all 2024-05-09 11:33:56 +08:00
Yanlong Wang
59f807cb7c
fix: tidyMarkdown 2024-05-09 11:32:26 +08:00
Yanlong Wang
6b6774f43b
fix: tidyMarkdown 2024-05-09 11:25:51 +08:00
Yanlong Wang
4bee36ed4a
fix: patch tidyMarkdown 2024-05-09 11:06:20 +08:00
Yanlong Wang
de22127d2f
fix: leak of crippled listeners 2024-05-08 19:51:55 +08:00
Han Xiao
b7523f0b2a
Update README.md 2024-05-08 13:30:28 +02:00
Yanlong Wang
62dc75f78e
fix: consider image data-src and make generated alt text optional (#50)
* fix: image src and alt

* fix

* docs: doc about x-with-generated-alt

* fix: deps
2024-05-08 18:29:11 +08:00
Yanlong Wang
8cfd0d67dc
feat: jina paywall (#49)
* feat: integrate with jina embeddings paywall
2024-05-08 18:25:26 +08:00
Yanlong Wang
2e025d10cf
fix: the complex regexp caused node.js process to hang
Co-authored-by:  Claude 3 opus
2024-05-05 16:29:39 +08:00
Yanlong Wang
fef1d0faf1
bump: deps 2024-05-05 10:54:11 +08:00
Han Xiao
e0a9025b4f
Update README.md 2024-05-03 20:41:51 +02:00
Yanlong Wang
3bb7bd1c29
chore: bump deps 2024-05-04 00:46:15 +08:00
Yanlong Wang
a0d1a7234b
chore: tweak health check 2024-05-02 08:39:54 +08:00
Yanlong Wang
9e02080103
fix: error on browser crashes 2024-05-02 03:23:57 +08:00
Yanlong Wang
55b954ffeb
fix: tweak health check 2024-04-30 18:56:46 +08:00
Yanlong Wang
528b3e5fed
fix: add health check to detect puppeteer stall 2024-04-30 18:30:31 +08:00
Yanlong Wang
ae29055142
chore: tweaks 2024-04-29 20:12:11 +08:00
yanlong.wang
867636d037
fix: apply rate limit to 100qpm per IP 2024-04-29 18:54:51 +08:00
yanlong.wang
15606f38d7
fix: on null element 2024-04-29 17:28:07 +08:00
yanlong.wang
53a4361c23
fix: block firebase runtime intrusion 2024-04-29 17:21:34 +08:00
yanlong.wang
059c8aa61e
fix: remove exposed function before cleanup 2024-04-29 15:51:23 +08:00
yanlong.wang
bfc6d678d8
fix: split report handler from other page preps 2024-04-29 15:19:05 +08:00
Yanlong Wang
036f6dc776
chore: tweak runtime config 2024-04-29 09:49:29 +08:00
Yanlong Wang
6ac2863e89
bump: deps 2024-04-28 22:28:24 +08:00
Han Xiao
fcd471b099
Update README.md 2024-04-25 20:35:28 +02:00
yanlong.wang
a6a5b7c530
fix: respond with markdown 2024-04-25 18:58:42 +08:00
yanlong.wang
69231ad59e
feat: full markdown mode 2024-04-25 18:21:04 +08:00
Han Xiao
0f70723d19
Update README.md 2024-04-25 10:10:32 +02:00
yanlong.wang
adc05fe20a
fix 2024-04-25 16:09:23 +08:00
Han Xiao
c2cd71d6a5
Update README.md 2024-04-25 10:06:24 +02:00
yanlong.wang
39a446f5e7
fix: root content 2024-04-25 15:43:17 +08:00
yanlong.wang
f1016649ac
fix: firebase limit on document size causing cache failures 2024-04-25 12:24:05 +08:00
Han Xiao
9dd5af0cb5
Update README.md 2024-04-24 17:30:08 +02:00
Han Xiao
ae788c39c5
docs: document header usage 2024-04-24 17:28:55 +02:00
yanlong.wang
94a72052f4
fix: reduce frequency of screenshot if possible 2024-04-24 19:43:24 +08:00
yanlong.wang
ae99af50aa
Merge branch 'main' of github.com:jina-ai/url2text 2024-04-24 19:21:50 +08:00
yanlong.wang
230388529e
bump: deps 2024-04-24 19:21:44 +08:00
Yanlong Wang
7ee2c327a3
refactor: reorganize features (#37)
* wip

* fix

* wip

* cleanup

* fix

* fix

* cache: may rescue using stale cache

* fix: target 384mb ram per page

* fix: log about pool size

* fix

* clean

* fix: cache and snapshot reporting
2024-04-24 19:21:12 +08:00
dependabot[bot]
e36d3b0f24
chore(deps): bump protobufjs and firebase-admin in /backend/functions (#35)
Bumps [protobufjs](https://github.com/protobufjs/protobuf.js) to 7.2.6 and updates ancestor dependency [firebase-admin](https://github.com/firebase/firebase-admin-node). These dependencies need to be updated together.


Updates `protobufjs` from 7.2.4 to 7.2.6
- [Release notes](https://github.com/protobufjs/protobuf.js/releases)
- [Changelog](https://github.com/protobufjs/protobuf.js/blob/master/CHANGELOG.md)
- [Commits](https://github.com/protobufjs/protobuf.js/compare/protobufjs-v7.2.4...protobufjs-v7.2.6)

Updates `firebase-admin` from 11.11.1 to 12.1.0
- [Release notes](https://github.com/firebase/firebase-admin-node/releases)
- [Commits](https://github.com/firebase/firebase-admin-node/compare/v11.11.1...v12.1.0)

---
updated-dependencies:
- dependency-name: protobufjs
  dependency-type: indirect
- dependency-name: firebase-admin
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-24 16:37:38 +08:00
yanlong.wang
c5bc474964
cleanup: remove top level package lock 2024-04-24 16:34:14 +08:00
Yanlong Wang
4b208f44b5
fix: process not quitting on errors 2024-04-21 10:17:05 +08:00
Han Xiao
17415ed1f1
docs: fix readme image 2024-04-20 23:27:42 +02:00
Yanlong Wang
5d255dda3b
chore: update deps 2024-04-19 09:30:19 +08:00
Charuka Samarakoon
d47310a6f7
fix: allocating incorrect max value due to missing parentheses (#26) 2024-04-19 09:01:23 +08:00
yanlong.wang
d4ca381c38
fix: explicitly reject non http protocols 2024-04-18 15:35:06 +08:00
yanlong.wang
abc817e960
feat: block media resources to improve speed 2024-04-18 15:06:28 +08:00