Web scraping of computer prices

Buffer Overflow
1

Hallo Zusammen ich bin ein Totaler Anfänger im web scraping. ich brauche hilfe von euch da ich ein Http Code nicht richtig lesen kann.

// ==/UserScript==

(function() {
    'use strict';

    window.setTimeout(installButton, 1000)

    function installButton() {
        const elem = document.createElement('button')
        elem.innerText = 'Copy'
        elem.style = 'z-index: 1000; position:fixed; top:0; left:0';
        document.body.appendChild(elem);
        elem.addEventListener('click', _ => exportCsv())
    }

    function exportCsv() {
        let products = Array.prototype.map.call(document.querySelectorAll("div[data-test='mms-search-srp-productlist-item'"), e => e)
        let csv = products.map(productObject).map(p => '"' + p.title + '",' + p.price + ',"' + p.url + '"').join('\n')
        console.log(csv)
        navigator.clipboard.writeText(csv)
            .then(_ => window.alert('In Zwischenablage kopiert'),
                  _ => window.alert('Fehler beim Kopieren in Zwischenablage'))
    }

    function productObject(e) {
        const result = {};
   
	**result.price = e.querySelector("div[**********]").innerText**
	result.url = e.querySelector('a').href
        result.title = e.querySelector('p').innerText.trim()




        return result
    }
})();

„LENOVO Yoga S740, Notebook mit 14 Zoll Display, Core i5 Prozessor, 8 GB RAM, 512 GB SSD, Intel Iris Plus Grafik, Mica/Champagne Glimmer“,„https://www.mediamarkt.de/de/product/_lenovo-yoga-s740-2589560.html“

Leider bekomme ich den Preis nicht raus ?
(**********)

https://www.mediamarkt.de/de/category/_ultrabooks-462727.html

Evtl könnte mir jemand dabei helfen.

Vielen dank schon mal im Vorraus

2

Hi @nexuz89 and welcome to the forum. As IT enthusiast, I appreciate your interest in keeping track of hardware prices. It would be nice if MediaMarkt and other shops provided automated tools to keep track of prices - and some already do: Digitec, for example, sends an email notification when the price of an item I’m interested in changes. Some portals like PC Part Picker and TopPreise offer this kind of thing. But an API or open format feed that would let us build our own tools without having to scrape the website is the best scenario.

I’m not sure exactly what the problem is you’re experiencing, but I can see you are reading directly from the DOM, which leads me to believe that the website has protections in place to make you have to use JavaScript in the browser runtime to collect the data - rather than plain HTML, which would be much easier to do with something like Scrapy.

Have a look at Stack Overflow’s scraping tag for ideas of other approaches, and hope you manage to purchase your dream laptop soon :slight_smile: