7 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
rmgr	f4ea8ad1d7	Respect robots.txt	2024-01-01 19:53:22 +10:30
rmgr	b43343e0ee	Fix recursive crawling.	2023-12-12 17:35:15 +10:30
rmgr	21961fced6	Add notes to index.py	2023-12-06 08:31:54 +10:30
rmgr	9fc2e1af53	Implement recursive page crawling	2023-12-06 08:29:39 +10:30
rmgr	3d7b72e5ef	Join counts for multiple words	2023-11-30 17:26:59 +10:30
rmgr	d30397cefa	Add count of times word appears on a site to index.	2023-11-30 08:03:43 +10:30
rmgr	f36ab2fbfb	Initial commit	2023-11-28 20:51:54 +10:30