[mralihaider]

hello, i want to copy all pages url links from shutterstock website in the food category

www.shutterstock.com/search?page=1&searchterm=Food&sort=popular&image_type=photo&safe=true&search_source=base_search_form&language=en

note that the number in red is a page number variable and it is from 1 until 151018
so {1..151018}

what tool can i use, if you could tell me a tool
or write me bash script i would appreciate it.

[SIM-3642159]

im not going to write the script for you but, wget the url, sed to remove the urls and output them in a txt file and loop my dude.

[Hackoo]

Hi ✌ if you mean a batch file 😅
Just a little Loop For /L and you will got it like this script 😉

Get_shutterstock_Urls.bat
Get_shutterstock_Urls.bat (Click to View)

Code

@echo off
Title Copying website pages url links from shutterstock by Hackoo 2017
mode con:cols=100 lines=3 & COLOR 0E
Set "URL_File=shutterstock_URLS.txt"
If Exist "%URL_File%" Del "%URL_File%"
echo(
echo(
For /L %%i in (1,1,151018) Do (
    Cls & echo(
    echo    Please Wait a while Copying website pages url links %%i from shutterstock is in progress ...
    (
        echo http://www.shutterstock.com/search?page=%%i^&searchterm=Food^&sort=popular^&image_type=photo^&safe=true^&search_source=base_search_form^&language=en
    )>>"%URL_File%"
)
Start "" "%URL_File%"

---

Edit : You can also do it, with a vbscript file, it's much faster then the batch file 😉 😝

Get_shutterstock_Urls.vbs
Get_shutterstock_Urls.vbs (Click to View)

PHP Code


<?php 
Option Explicit
Dim fso,ws,File,URL_File,i
Set fso = CreateObject("Scripting.FileSystemObject") 
Set ws = CreateObject("WScript.Shell")
URL_File = "shutterstock_URLS.txt"
Set File = fso.OpenTextFile(URL_File,2,True)
For i=1 to 151018
    File.WriteLine "http://www.shutterstock.com/search?page="& i &_
    "&searchterm=Food&sort=popular&image_type=photo&safe=true&search_source=base_search_form&language=en"
Next
ws.Run "Notepad "& URL_File 


👍 😉 👍

[mralihaider]

thank you Hackoo but i use linux mint
i can't use your script
still need this question if you could help i would appreciate it

[oneghost]

lt's sound for me as a scraper your questions if I understood it wrong please send me more details on PM ( what info do you want to gather , url pictures description?)
you should try to find which one will be the last page before increment the page number and when the last page will be = with the value from loop to make the script to stop.
with curl , sed , grep and a loop you may be able to do it.
or if you didn't decide yet how to do it, I may recommend you to use php (with dom or directly CURL with some functions ) or perl scripting , as you may randomize the information's about your browser.