+- Hack Forums (https://hackforums.net)
+-- Forum: Tech (https://hackforums.net/forumdisplay.php?fid=444)
+--- Forum: Coding (https://hackforums.net/forumdisplay.php?fid=151)
+---- Forum: Software Development (https://hackforums.net/forumdisplay.php?fid=118)
+---- Thread: [Python] Img link grabber (/showthread.php?tid=5089890)
Img link grabber - emadaldin1 - Dec 11, 2015
Hello everybody
I need script to pull all img links from site just like google images
the problem you might face is
that the rest of the images
when you scroll down a new images will show up
etc
and after it grab all img links from the html or the page
store it on a file
then on linux
for each link on the text
open firefox + link of the image
that it
hope my thread is clear
summery
I need scripts that grab img links from website
you can give me an example and i will modify it
as long as you explain it well :D
note I'm using Ubuntu and Windows
* - sfdemo - Dec 11, 2015
below are the steps to extract the links as the rest are relatively simple for you to google
1)use selenium to visit the site
2)grab all the image links using css selector and put them in a set
3)then use selenium to simulate the END button keypress and repeat step 2 again
Note you can run them in a loop and stop after 30 runs or so. So to sum it up you need knowledge of 1) python 2) css selector 3) selenium
* - emadaldin1 - Dec 11, 2015
Well your replay was going to be perfect
If you provided an example source code
pls provide a source code for your replay
I clearly asked for python script
Thanks
I will also try your answer for now
* - Helltoy - Dec 11, 2015
You can try with BeautifulSoup and requests.
This is just one example, how I grab specific image from Steam.
Python 2.7
Code
import requests
from bs4 import BeautifulSoup
page = requests.get(steam_game_link)
soup = BeautifulSoup(page.content , 'lxml')
background_imgs = soup.find_all('img', {'class':'game_header_image_full'})
for background_img in background_imgs:
background_img = background_img.get('src')I think if you make some changes, you are able to grab all links from page.
* - sfdemo - Dec 11, 2015
(Dec 11, 2015 08:46 AM)Helltoy Wrote: You can try with BeautifulSoup and requests.
This is just one example, how I grab specific image from Steam.
Python 2.7
Codeimport requests
from bs4 import BeautifulSoup
page = requests.get(steam_game_link)
soup = BeautifulSoup(page.content , 'lxml')
background_imgs = soup.find_all('img', {'class':'game_header_image_full'})
for background_img in background_imgs:
background_img = background_img.get('src')
I think if you make some changes, you are able to grab all links from page.
note he has to scroll down such that new pages are able to load. i don't think you can do that with bs4.
my method definitely works since i have done that for pinterest and it is the same as what he describes
* - Kevn - Dec 11, 2015
Use firefox and download DownThemAll here is the link
https://addons.mozilla.org/nl/firefox/addon/downthemall/