custom crawler

Content crawling - Amazon

What I need to think about, when I am programming an application, which will download the content from Amazon and store it in a database. I have a program, which has his own web crawler for downloading web content. The problem is that this program cannot recognize the important parto of every webpage and we need to somehow help with important part recognition. This issue is important to solve for Amazon webpage.
Subscribe to custom crawler