[Freed0m36]

I am writing a python web scraper which will use a document with a list of url's to scrape,
scrape the data I want and then write it to a file.
The list contains 24 url's and I am using "sleep" a few time in my functions to make sure not to overflow the websites with requests..
I want to implement a progress bar like "TQDM". but I am not sure how to use it.
Everywhere I look, they give examples with "for" loops and I'm not sure where to place that loop in my code to have a Global progress overview.

If anyone has experience with TQDM ( or any other progress bar module ) and the skills to solve my problem,
any help will be appreciated.

[Qismon]

This should work fine.

Code

import progressbar
from time import sleep

bar = progressbar.ProgressBar(maxval=100, \
    widgets=[progressbar.Bar('=', '[', ']'), ' ', progressbar.Percentage()])
bar.start()
for i in xrange(100):
    bar.update(i+1)
    sleep(0.1)
bar.finish()

[DopplerAF]

I don't have clear idea about what you are asking about but I have done something which might help you.

Code

from bs4 import BeautifulSoup
from urllib import request
import time
from progressbar import ProgressBar, Bar, Percentage

pbar = ProgressBar(widgets=[Percentage(), Bar()], maxval=100)

content = request.urlopen("https://www.youtube.com").read()
soup = BeautifulSoup(content, features="html.parser")
test_file = open("test.txt", "w")

for i in pbar(soup.find_all('a')):
    test_file.write(str(i.get("href")) + "\n")
    time.sleep(0.05)
test_file.close()

[PoP-Hacker]

import time
import sys

toolbar_width = 40


sys.stdout.write("[%s]" % (" " toolbar_width))
sys.stdout.flush()
sys.stdout.write("\b" (toolbar_width+1))

for i in range(toolbar_width):
time.sleep(0.1)

sys.stdout.write(">")
sys.stdout.flush()

sys.stdout.write("\n")

[StreetSquirrel]

import time
import sys

toolbar_width = 40


sys.stdout.write("[%s]" % (" " toolbar_width))
sys.stdout.flush()
sys.stdout.write("\b" (toolbar_width+1))

for i in range(toolbar_width):
   time.sleep(0.1)
   
   sys.stdout.write(">")
   sys.stdout.flush()

sys.stdout.write("\n")

[*code] [*/code]

[PoP-Hacker]

[*code] [*/code]

?

[Infamia]

import time
import sys

toolbar_width = 40


sys.stdout.write("[%s]" % (" " toolbar_width))
sys.stdout.flush()
sys.stdout.write("\b" (toolbar_width+1))

for i in range(toolbar_width):
   time.sleep(0.1)
   
   sys.stdout.write(">")
   sys.stdout.flush()

sys.stdout.write("\n")

unnecessarily using `sys.stdout.write` is pointless.

[PoP-Hacker]

unnecessarily using `sys.stdout.write` is pointless.

HAHA You not well done me but small problem in replay me