mardi 1 décembre 2020

Writing crawler in ruby

I am writing a crawler in ruby, for the movie section of bsvalue website. https://bsvalue.com/category/5f74877a50e0c79d5d58b8a4;categoryTitle=Movies I am able to get the json, but i don't have idea how to save/print that all the movies name , id, cast etc, one by one using parmas . Can anyone help me in this.

here is the starting of my code which gives me json response

require "net/http"

url = URI("https://blacksheepapi.com/app-v2/api/mobile/home/category/5f74877a50e0c79d5d58b8a4/with-videos?offset=0&limit=40")

https = Net::HTTP.new(url.host, url.port)
https.use_ssl = true

request = Net::HTTP::Get.new(url)
request["authority"] = "blacksheepapi.com"
request["pragma"] = "no-cache"
request["cache-control"] = "no-cache"
request["accept"] = "application/json, text/plain, */*"
request["authorization"] = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJkYXRhIjp7ImlkIjoiNWZhM2EzYWZmNmRhY2MxMjVkOTQ3ZDlmIiwidXNlclR5cGUiOjIsImlzUHJlbWl1bVVzZXIiOmZhbHNlfSwiaWF0IjoxNjA2NzMxMTAwLCJleHAiOjE2MDkzMjMxMDB9.gv7iOF8s6aKBqiTMw1q-mdkxRjnfPnivuG4orBg2T0c"
request["user-agent"] = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.67 Safari/537.36"
request["origin"] = "https://bsvalue.com"
request["sec-fetch-site"] = "cross-site"
request["sec-fetch-mode"] = "cors"
request["sec-fetch-dest"] = "empty"
request["referer"] = "https://bsvalue.com/"
request["accept-language"] = "en-GB,en-US;q=0.9,en;q=0.8"

response = https.request(request)
puts response.read_body

json = JSON.parse(response.body)

This is the json response I'm getting

    "isVisible" => true,
    "isDeleted" => false,
    "videoIds" => [{
        "isPremiumVideo" => false,
        "ageLimit" => 6,
        "_id" => "5fa01e17d7f2c12af4b03cf8",
        "title" => "Anjaneya",
        "images" => {
            "_id" => "5fb3b34f367224583e2f1ff4", "mobileBannar" => "5fa01e17d7f2c12af4b03cf8/assets/450_270.jpg?160561236759778053", "mobileThumbMedium" => "5fa01e17d7f2c12af4b03cf8/assets/370_500.jpg?160561236759790246", "tvDetailBannar" => "5fa01e17d7f2c12af4b03cf8/assets/1280_720.jpg?160561236759773244", "videoLogo" => "5fa01e17d7f2c12af4b03cf8/assets/600_200.png?160561236759760443", "posterUrl" => "5fa01e17d7f2c12af4b03cf8/assets/video-poster.jpg?160561236759725158"
        },
        "cast" => "Ajith Kumar, Meera Jasmine, Raghuvaran, Seetha, Jaya Prakash Reddy.",
        "categoryId" => "5f74877a50e0c79d5d58b8a4",
        "description" => "ACP Paramaguru (Ajith Kumar) is an efficient police officer who fights with the scum of the society. He later masquerades as a thief to infiltrate into the underworld. The bad guys are surprised to find that Paramguru is the ACP and is out to get them, so they all gang up against him. In this process, a thrilling encounter takes place between the good and the evil. Paramaguru is helped in his fight against injustice by Divya (Meera Jasmine), who falls in love with him.\n\nAnjaneya is a 2003 Indian Tamil-language action film directed by N. Maharajan. ",
        "miniPreviewUrl" => "5fa01e17d7f2c12af4b03cf8/assets/mini_preview.mp4",
        "sourceVideoQuality" => "1080p",
        "timeCode" => "2hr 54min 18sec",
        "videoType" => "ORIGINAL"
    }, {
        "isPremiumVideo" => false,
        "ageLimit" => 6,
        "_id" => "5fa29daad7f2c12af4b0adba",
        "title" => " Agni Natchathiram",
        "images" => {
            "_id" => "5fb3b35c367224583e2f2d6b", "mobileBannar" => "5fa29daad7f2c12af4b0adba/assets/450_270.jpg?160561238048172958", "mobileThumbMedium" => "5fa29daad7f2c12af4b0adba/assets/370_500.jpg?160561238048143726", "tvDetailBannar" => "5fa29daad7f2c12af4b0adba/assets/1280_720.jpg?160561238048193416", "videoLogo" => "5fa29daad7f2c12af4b0adba/assets/600_200.png?160561238048128347", "posterUrl" => "5fa29daad7f2c12af4b0adba/assets/video-poster.jpg?160561238048142775"
        },
        "cast" => "Prabhu , Karthik , Amala",
        "categoryId" => "5f74877a50e0c79d5d58b8a4",
        "description" => "Agni Natchathiram (transl. Scorching Star) is a 1988 Indian Tamil-language action drama film written and directed by Mani Ratnam. The film stars Prabhu, Karthik, Amala and newcomer Nirosha, with Vijayakumar, Jayachitra, Sumithra, Tara, S. N. Lakshmi and G. Umapathy in pivotal roles. It revolves around two half-brothers who come into conflict with each other in their claim for legitimacy as sons of a common father.",
        "miniPreviewUrl" => "5fa29daad7f2c12af4b0adba/assets/mini_preview.mp4",
        "sourceVideoQuality" => "1080p",
        "timeCode" => "2hr 19min 23sec",
        "videoType" => "ORIGINAL"
    }, {
        "isPremiumVideo" => false,
        "ageLimit" => 6,
        "_id" => "5fa670c76785a54d2d96f9d5",
        "title" => " Vaanchinathan",
        "images" => {
            "_id" => "5fb3b365367224583e2f3968", "mobileBannar" => "5fa670c76785a54d2d96f9d5/assets/450_270.jpg?16056123899617075", "mobileThumbMedium" => "5fa670c76785a54d2d96f9d5/assets/370_500.jpg?160561238996146991", "tvDetailBannar" => "5fa670c76785a54d2d96f9d5/assets/1280_720.jpg?160561238996144346", "videoLogo" => "5fa670c76785a54d2d96f9d5/assets/600_200.png?160561238996173272", "posterUrl" => "5fa670c76785a54d2d96f9d5/assets/video-poster.jpg?160561238996137035"
        },
        "cast" => "vijaykanth , ramyakrishnan",
        "categoryId" => "5f74877a50e0c79d5d58b8a4",
        "description" => "Vanchinathan (Vijayakanth) is naturally the supercop who has been transferred from Gujarat. He is someone who is not averse to using the law in his own ways by killing someone and picking a suitable alibi, as long as the person is evil and will not be punished by the law.",
        "miniPreviewUrl" => "5fa670c76785a54d2d96f9d5/assets/mini_preview.mp4",
        "sourceVideoQuality" => "1080p",
        "timeCode" => "2hr 47min 55sec",
        "videoType" => "ORIGINAL"
    }, {
        "isPremiumVideo" => false,
        "ageLimit" => 6,
        "_id" => "5fa6719d6785a54d2d96f9d8",
        "title" => " Periyanna",
        "images" => {
            "_id" => "5fb3b370367224583e2f46c3", "mobileBannar" => "5fa6719d6785a54d2d96f9d8/assets/450_270.jpg?160561240070856606", "mobileThumbMedium" => "5fa6719d6785a54d2d96f9d8/assets/370_500.jpg?160561240070824901", "tvDetailBannar" => "5fa6719d6785a54d2d96f9d8/assets/1280_720.jpg?160561240070872441", "videoLogo" => "5fa6719d6785a54d2d96f9d8/assets/600_200.png?160561240070832853", "posterUrl" => "5fa6719d6785a54d2d96f9d8/assets/video-poster.jpg?160561240070845895"
        },
        "cast" => "surya , meena",
        "categoryId" => "5f74877a50e0c79d5d58b8a4",
        "description" => "periyana",
        "miniPreviewUrl" => "5fa6719d6785a54d2d96f9d8/assets/mini_preview.mp4",
        "sourceVideoQuality" => "1080p",
        "timeCode" => "2hr 22min 42sec",
        "videoType" => "ORIGINAL"
    }, {
        "isPremiumVideo" => false,
        "ageLimit" => 6,
        "_id" => "5fa924d16785a54d2d96fd64",
        "title" => "Villain",
        "images" => {
            "_id" => "5fb3b37b367224583e2f534c", "mobileBannar" => "5fa924d16785a54d2d96fd64/assets/450_270.jpg?160561241118737975", "mobileThumbMedium" => "5fa924d16785a54d2d96fd64/assets/370_500.jpg?160561241118769411", "tvDetailBannar" => "5fa924d16785a54d2d96fd64/assets/1280_720.jpg?160561241118738963", "videoLogo" => "5fa924d16785a54d2d96fd64/assets/600_200.png?160561241118799290", "posterUrl" => "5fa924d16785a54d2d96fd64/assets/video-poster.jpg?160561241118758637"
        },
        "cast" => " Ajith & Meena,Kiran. Sujatha, FEFSI Vijayan, Karunaas , Ramesh Khanna",
        "categoryId" => "5f74877a50e0c79d5d58b8a4",
        "description" => "\nVillain is a 2002 Indian Tamil-language action masala film written and directed by K. S. Ravikumar and produced by S. S. Chakravarthy",
        "miniPreviewUrl" => "5fa924d16785a54d2d96fd64/assets/mini_preview.mp4",
        "sourceVideoQuality" => "1080p",
        "timeCode" => "2hr 44min 36sec",
        "videoType" => "ORIGINAL"
    }],```

Aucun commentaire:

Enregistrer un commentaire