update base_path in crawl

update joy and crawl UwU

Fix: Correct retry logic to prevent multiple restarts

Fix: Move retry_crawl function definition above its usage

dont redefine variables from outer scope + code formatting

feat: Add parallel execution for multiple URLs

retry

update crawl

trailing whitespace yay

handle multiple urls

fix downloading images when the domain is missing

Revert "fix: Add validation for image URLs to prevent invalid downloads"

Revert "fix: Enhance URL validation for image downloads- Improved the URL validation in the download_image function.- Used requests.utils.urlparse to parse the URL and check for a valid scheme and netloc.- Ensured the URL path does not contain invalid labels (e.g., ..).- This prevents the script from attempting to download images with invalid URLs and provides clearer error messages."

Revert "fix: Improve URL validation to catch invalid labels in image downloads"

fix: Improve URL validation to catch invalid labels in image downloads

fix: Enhance URL validation for image downloads- Improved the URL validation in the download_image function.- Used requests.utils.urlparse to parse the URL and check for a valid scheme and netloc.- Ensured the URL path does not contain invalid labels (e.g., ..).- This prevents the script from attempting to download images with invalid URLs and provides clearer error messages.

fix: Add validation for image URLs to prevent invalid downloads

Revert "Improve error handling in save_result function"

Improve error handling in save_result function

Initial commit