Douban crawler

Just as I mention in previous article, I extend my program to get datas from Douban site.

I start to collect some user data from group member list. It’s not hard, just need to wait for some interval because of timing-out protection mechanism. I got 21000 user datas after try few groups. It’s enough for me.

Another one is Movie data. I plan to get all the favorite movie from current users. I still try the ugly html. I think it’s good time for me to practice rescue keyword of Ruby.

I still need some test to see if I got robust script or not.

Still have many thing to do. And want to practice d3.js for display the relation of user and movie. And then, maybe try some impressive.js.

I saw there exist one website is using impressive.js for web design. Really impressive!! Maybe I will try to create one. :)

 
1
Kudos
 
1
Kudos

Now read this

Schedule

Today I do many things, and plan even more. I create one page in my website. Link. This link will trigger every time to retrieve the article in baidu forum. It will get the article title list in the page first, filter out non-sense and... Continue →