+- Hack Forums (https://hackforums.net)
+-- Forum: Tech (https://hackforums.net/forumdisplay.php?fid=444)
+--- Forum: Coding (https://hackforums.net/forumdisplay.php?fid=151)
+---- Forum: Web Development (https://hackforums.net/forumdisplay.php?fid=183)
+---- Thread: [PHP] (Release) Easy Scrapping of HTML Content.. (/showthread.php?tid=6227820)
(Release) Easy Scrapping of HTML Content.. - Shellcode™ - Nov 17, 2022
Hey!
So while working on countless scraper scripts, I have compiled my own little PHP Class to do the process as simple & handy as I want it to be.. :)
So why keep it for my-self, and not share my work? Hell yeah!
This will come in very handy for you, if you need a very quick way to do data scraping with minimal effort! If you have any questions or requests, please contact me.
/Scraper.php
https://rentry.co/daiub
Basic Example:
PHP Code
<?php
function PullTableData($url) {
if($content = $this->curl($url)) {
$listings_html = $this->getValueByTagName($content, '<div id="mainblock">', '<div class="leftblock">');
$listings_html_Arr = explode('<div class="lbm">', $listings_html);
array_shift($listings_html_Arr);
foreach($listings_html_Arr as $key => $data) {
// populate data with info returned by $data & $key..
$items[$key] = array(
'identifier' => 'paramter'
);
}
}
else {
echo 'Unable to load the URL Provided: ' .$url;
return false;
}
return $items;
}
Let me know if you need any further instructions on how to utilize the methods within this class.
RE: (Release) Easy Scrapping of HTML Content.. - DashPHP - Nov 18, 2022
Very cool stuff there buddy!
RE: (Release) Easy Scrapping of HTML Content.. - Hackers - Nov 19, 2022
Nice, great start!
One thing you should explore is xpath routes so you can more easily fetch data. It is faster than using explodes.
The regex, string searches, length checks, etc, may be slower than using something like Domxpath or a similar package. Your code though is impressive and I am glad it works and appears to be user friendly too.
I was making PHP crawlers back in 2015-18 and was able to power a lot of businesses with data because of it. Are you using your scripts for anything right now?