Home Upgrade Search Memberlist Extras Hacker Tools Award Goals Help Wiki Contact

Shell 

Pulling links from source code with grep?

Hey guys, first post

I've been taking online courses online for security+ and some other subjects, it's pretty much just video tutorials, I looked at the source code and the links to the videos aren't hidden so I can pull them down and save them, but there's loads of videos. I'm trying to run grep to pull the links from the source code but not having much luck.
cat source.php | grep *.mp4 > links.txt
Thats the command I tried but no luck, not sure if the command is wrong or if it's because the links aren't on their own line. Any help would be much appreciated :)
Code
cat source.php | grep -Eo '(http|https):\/\/[^"]+\.mp4' > links.txt
That's awesome thanks a lot :)
Better get my head back in the man page, I'm not as good as I'd like to be in the terminal yet
Save them all to the current working directory:

Code
for m in $(cat source.php | grep -Eo '(http|https):\/\/[^"]+\.mp4'); do
        curl -O $m;
done
Is that perl in there too? The more you reply the less I feel I know, what would you recommend as a good resource to master the shell?
I've got The Linux Command Line by No starch press and some RHCSA video tutorials but they don't seem to cover this to your extent
(May 27, 2017 12:26 PM)Tesseract.mk Wrote:  Is that perl in there too? The more you reply the less I feel I know, what would you recommend as a good resource to master the shell?
I've got The Linux Command Line by No starch press and some RHCSA video tutorials but they don't seem to cover this to your extent

No - it's simply Bash, with Regex for matching the URLs.

Got used to it by doing, never read an actual book on it: just man pages and Google.
I've worked more seriously with other scripting languages though, so that probably makes it easier.
If your are intersted with batch too; here is some examples : How to extract data from website ?


Possibly Related Threads...
Thread / Author Replies Views Last Post
by CaelrithOct 30, 2025 07:47 AM 0 73 Oct 30, 2025 07:47 AM
Last Post: Caelrith
[Shell] by joshythejew101Dec 29, 2020 07:57 PM 2 173 Dec 29, 2020 07:57 PM
Last Post: joshythejew101
[Python] by koolkidJan 17, 2018 12:57 PM 1 102 Jan 17, 2018 12:57 PM
Last Post: Anxuein