Hi I am trying to write a web scraper but I keep getting this error:
http://ift.tt/1QmM9QS
/Users/alexwang/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/uri/common.rb:176:in `split': bad URI(is not URI?): http://ift.tt/1QmM9QS (URI::InvalidURIError)
from /Users/alexwang/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/uri/common.rb:211:in `parse'
from /Users/alexwang/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/uri/common.rb:747:in `parse'
from /Users/alexwang/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/uri/common.rb:1232:in `URI'
the page I am trying to open up is http://ift.tt/1QmM9QS
I guess the error is caused by the '[id]' contained in the URL, when I manually remove it, it works. But it's not wise for me to do so because it's an intermediate step within a large spider. I tried the encode method mentioned in the other pose but it doesn't work for me.
Aucun commentaire:
Enregistrer un commentaire