부산대학교 도서관

Web crawlers have been widely used to collect data and information efficiently. This paper first analyzed the application of web crawlers in information crawling, introduced the topic crawler and HITS algorithm, and proposed an improved HITS algorithm to improve the crawling accuracy of web pages. The experimental analysis verified that the improved algorithm was reliable in enhancing the crawling accuracy and always had a crawling accuracy above 0.7 when crawling different topics, higher than the traditional HIS algorithm. Then, this paper analyzed the infringement of personal information by web crawlers, described some relevant cases, pointed out the shortcomings of existing laws, and gave some suggestions. This work supports further improving the rule system needed for crawler applications and punishing personal information infringement.