TY  - RPRT
TI  - Web crawler strategies for web pages under robot.txt restriction
AU  - Piyush Vyas
AU  - Akhilesh Chauhan
AU  - Tushar Mandge
AU  - Surbhi Hardikar
PY  - 2024
UR  - https://arxiv.org/abs/2308.04689
ID  - 2308.04689
ER  -