add function to scrape the upper bound of the search pagination before running main loop

add compiling instructions to readme
update .gitignore
This commit is contained in:
Neo committed 2023-03-31 01:58:51 -07:00
1 parent 1328f837ee
commit 3ebdc5dfbd
4 files changed
+65 -3

No files matched your search

+12
View File
@@ -1,2 +1,14 @@
# winworldpc-ipfs-scraper
grabs ipfs links for files from winworldpc
------------
no downloads are provided at this time
simply clone the project and run `go build`
running the resulting executable will begin the scraping process
progress is not saved and will restart from the first page, while appending to existing files
hopefully you found this project useful