Home Upgrade Search Memberlist Extras Hacker Tools Award Goals Help Wiki Contact

Python 

How to parse js with python

Hi i have a website
When i try to parse it with bs4 and search only for paragraphs it shows: <p id = "test"></p>
But there is jscript: document.getElementById("test").innerHTML = "It works";
and should show: It works
i have a code that "should simulate the browser" and show the right result, the problem is that the code doesnt work.

code:

from bs4 import BeautifulSoup as soup
import sys
import requests
from PyQt5.QtWebEngineWidgets import QWebEnginePage
from PyQt5.QtWebEngineWidgets import QWebEngineView
from PyQt5.QtWidgets import QApplication
from PyQt5.QtCore import QUrl
class Client(QWebEnginePage):
   def _init_(self, url):
       self.app = QApplication(sys.argv)
       QWebEnginePage.__init__(self)
       self.loadFinished.connect(self.on_page_load)
       self.mainFrame().load(QUrl(url))
       self.app.exec()

   def on_page_load(self):
       self.app.quit()
url = "test"
client_response = Client(url)
file = client_response.mainFrame().toHtml()

sp = soup(file, "html.parser")
print(sp.find_all("p"))

and it gives me this error:
 client_response = Client(url)
TypeError: arguments did not match any overloaded call:
 QWebEnginePage(parent: QObject = None): argument 1 has unexpected type 'str'
 QWebEnginePage(QWebEngineProfile, parent: QObject = None): argument 1 has unexpected type 'str'

i dont want to use selenium, because you should specify the browser

thanks and sorry for my bad english   🎉
Code
def _init_(self, url):

double under scores, like this:

Code
def __init__(self, url):
Contact for professional programming services.
unazed#5906
Thanks, i made it work but i have another problem

it parse it in the beggining, but when i change the value and run the code again it doesnt work

i have a button in the html, and when i press it i want to change the output, but the python code doesnt catch it

this is working
<p class = "jstest" id = "isactive" >Doesnt work</p>
<script> document.getElementById("isactive").innerHTML = "it works";
</script>

but this part - not :

function toggle(){
       if(!toggled){
           
           toggled = true;
           st.status = "On";
           document.getElementById("isactive").innerHTML = st.status;
return;

        }
.
.
.


Possibly Related Threads...
Thread / Author Replies Views Last Post
[Python] by Bass_ReevesAug 20, 2023 01:44 AM 9 632 Aug 20, 2023 01:44 AM
Last Post: Lucky4L
[Python] by XoggApr 24, 2020 01:14 AM 2 139 Apr 24, 2020 01:14 AM
Last Post: Lindros
[Shell] by ProfitableFlipMay 24, 2019 09:40 AM 1 163 May 24, 2019 09:40 AM
Last Post: Adolf