Book a Demo!
CoCalc Logo Icon
StoreFeaturesDocsShareSupportNewsAboutPoliciesSign UpSign In
mikf
GitHub Repository: mikf/gallery-dl
 Path: blob/master/CHANGELOG.md
5399 views

Changelog

1.30.5 - 2025-08-24

Extractors

Additions

  • [shimmie2] support noz.rip/booru (#8101)

  • [sizebooru] add support (#7667)

  • [twitter] add highlights extractor (#7826)

  • [twitter] add home extractor (#7974)

Fixes

  • [aryion] fix pagination (#8091)

  • [rule34] support using api-key & user-id (#8077 #8088 #8098)

  • [tumblr:search] fix ValueError: not enough values to unpack (#8079)

  • [twitter] handle KeyError: 'result' for retweets (#8072)

  • [zerochan] expect 500 Internal Server Error responses for HTML requests (#8097)

Improvements

  • [civitai:search] add token option (#8093)

  • [instagram] warn about lower quality video downloads (#7921 #8078)

  • [instagram] remove candidates warning (#7921 #7989 #8071)

  • [oauth] improve error messages (#8086)

  • [pixiv] distinguish empty from deleted profiles (#8066)

  • [twitter] update API endpoint query hashes & parameters

Metadata

  • [batoto] extract more metadata (#7994)

  • [instagram:highlights] extract author & owner & user metadata (#7846)

  • [newgrounds] extract slug metadata (#8064)

  • [twitter] extract community metadata (#7424)

Removals

  • [shimmie2] remove sizechangebooru.com (#7667)

  • [zzup] remove module (#4604)

Downloaders

  • [ytdl] improve playlist handling (#8085)

Scripts

  • implement rm helper script

  • add -g/--git command-line options

  • [util] add git() & lines() helper functions

Miscellaneous

  • [config] add conf argument to config.load() (#8084)

1.30.4 - 2025-08-16

Extractors

Additions

  • [civitai] add 'videos' extractor (#6644)

Fixes

  • [civitai] fix posts not returning video files (#8053)

  • [civitai] fix '403 Forbidden' errors for searches

  • [kemono] use 'Accept: text/css' for API requests (#8047 #8057)

Improvements

  • [newgrounds] add fallback for images with empty 'full_image_text' 'src'

Miscellaneous

  • fix accessing methods through 'path-metadata' proxy (#6582)

1.30.3 - 2025-08-15

Extractors

Additions

  • [booth] add support (#7920)

  • [civitai] add collection & user-collections extractors (#8005)

  • [facebook] add info extractor (#6582)

  • [facebook] add albums extractor (#7848)

  • [imgdrive] add image extractor (#7976)

  • [imgtaxi] add image extractor (#8019)

  • [imgwallet] add image extractor (#8021)

  • [picstate] add image extractor (#7946)

  • [silverpic] add image extractor (#8020)

  • [tumblr] add following & followers extractors (#8018)

  • [xasiat] add support (#4161 #5929 #7934)

Fixes

  • [blogger] fix video extraction (#7892)

  • [comick] handle chapters without chapter data (#7972)

  • [comick] handle volume-only chapters (#8043)

  • [comick] fix exception when filtering by translation group (#8045)

  • [deviantart:tiptap] fix KeyError: 'attrs' (#7929)

  • [everia] fix image extraction (#7973 #7977)

  • [facebook] fix avatar extraction for empty profiles (#7962)

  • [facebook] handle profiles without photos or set_id (#7962)

  • [fappic] rewrite thumbnail URLs (#8013)

  • [idolcomplex] update to new domain and interface (#7559 #8009)

  • [kemono][coomer] fix extraction (#8028 #8031)

  • [kemono] update /creators endpoint (#8039 #8040)

  • [kemono] don't set error status for posts without comments (#7961)

  • [pixiv] fix IndexError for unviewable works (#7940)

  • [pixiv] fix artworks downloads when using expired cookies (#7987)

  • [scrolller] fix NSFW subreddit pagination (#7945)

  • [twitter] fix potential UnboundLocalError when videos are disabled (#7932)

  • [vsco] disable TLS 1.2 cipher suites by default (#7984 #7986)

  • [wikimedia:wiki] fix AttributeError: 'subcategories' (#7931)

Improvements

  • [aibooru] support general.aibooru.online & aibooru.download

  • [comick] add lang option (#7938)

  • [hentaifoundry] add descriptions option (#7952)

  • [facebook] raise AuthRequired for profiles requiring cookies (#7962)

  • [instagram] warn about lower quality image downloads (#7921)

  • [kemono] support "endpoint": "posts+" for full metadata (#8028)

  • [misskey] support misskey.art (#7923)

  • [motherless] detect 404/File not found pages

  • [pixiv] detect suspended/deleted accounts (#7990)

  • [pixiv] improve API error messages

  • [pixiv] remove redundant cookies initialization code

  • [scrolller] limit title length in default filenames

  • [skeb] implement include option (#6558 #7267)

  • [vk] update default archive_fmt (#8030)

Metadata

  • [cien] provide author[id] metadata (#6582)

  • [dankefuerslesen] extract more metadata (#7915)

  • [dankefuerslesen:manga] fix metadata being overwritten

  • [facebook] ensure numeric user_id values (#7953)

  • [facebook:set] fix/improve user_id extraction (#7848)

  • [fappic] fix filename values

Common

  • [common] implement "user-agent": "@BROWSER" (#7947)

  • [common] improve error message for non-Netscape cookie files (#8014)

Downloaders

  • [ytdl] don't overwrite existing filename data (#7964)

Miscellaneous

  • [docs/configuration] improve client-id & api-key instructions

  • [docs/formatting] update and improve

  • [job] apply extension-map to SimulationJob results (#7954)

  • [job] improve URL scheme extraction performance

  • [job] split collected DataJob results

  • [path] implement path-convert option (#493 #6582)

  • [scripts] improve and extend init, generate_test_result, and pyprint

  • extend -A/--abort & "skip": "abort" functionality (#7891)

  • use more f-strings (#7671)

1.30.2 - 2025-07-27

Extractors

Additions

  • [itaku] add posts & bookmarks extractors (#7707)

Fixes

Post Processors

  • [exec] use False as start_new_session default to avoid a TypeError (#7899)

Miscellaneous

  • [tests/postprocessor] fix TypeError when logging an error (#6582)

1.30.1 - 2025-07-26

Extractors

Additions

  • [civitai] add generated extractor (#7796)

  • [facebook] add avatar extractor (#7848)

  • [imgadult] add image extractor (#7893)

  • [itaku] add following & followers extractors (#7707)

  • [leakgallery] add support (#7872)

  • [madokami] add manga extractor (#7828)

Changes

  • [civitai] change default user includes to ["user-images", "user-videos"] (#7874)

Fixes

  • [behance] fix 403 Forbidden errors by using "browser": "firefox" (#7803 #7877)

  • [civitai] fix AttributeError when a file's post was deleted (#7860)

  • [pornhub] fix gallery extractor (#7842)

  • [readcomiconline] force One page reading mode (#7890)

  • [sexcom] update search extractor (#7807)

  • [urlgalleries] fix extraction (#7858)

  • [wikimedia] add missing self argument when calling prepare() (#7835)

Improvements

  • [4chan] detect files containing only null bytes (#7883)

  • [azurelanewiki] prevent Anubis challenge

  • [bilibili] warn about blocked articles (#7880)

  • [civitai] fix extension for videos without name and mimeType

  • [common] detect Cloudflare & DDoS-Guard challenge pages in request_json() & request_xml() (#7833)

  • [facebook] add retries to profile page requests (#7725 #7834 #7852)

  • [facebook] implement include option (#7848)

  • [itaku] implement include option (#7707)

  • [patreon] implement cursor support (#7856)

  • [patreon] support date-max for /home URLs (#7856)

  • [pixiv] improve AJAX error messages (#7896)

Metadata

  • [behance] provide creator[name] metadata (#7885)

  • [civitai] ensure file & post data has a date value (#7548)

  • [inkbunny] enable pool metadata (#7850)

  • [nhentai] provide gallery_id for pagination results (#7868)

Downloaders

  • [ytdl] add deprecations option

Post Processors

  • [exec] add session option (#6582)

Snap

  • migrate base to core22 (#7841)

  • switch to yt-dlp (#7865)

  • fix deprecated CRAFT_ARCH_TRIPLET usage (#7866)

Formatter

  • add Jinja template support (#1390)

  • add W conversion - sanitize whitespace (#6582)

Miscellaneous

  • [actions] fix parse_logging import (#7837)

  • [options] add --sleep-429 command-line option (#7871)

  • [scripts] ensure files use utf-8 encoding and \n newlines (#7872)

  • [tests/extractor] improve example URL mismatch error message (#7872)

  • [tests/results] fix #log checks for URLs raising exceptions

  • fix exit status for requests' JSONDecodeError (#4380)

  • use walrus operators := in if statements (#7671)

1.30.0 - 2025-07-15

Changes

  • raise minimum supported Python version to 3.8 (#7671)

  • update extractor names (#7746 #7612)

    • OldNew
      kemonopartykemono
      coomerpartycoomer
      koharuschalenetwork
      navernaver-blog
      chzzknaver-chzzk
      naverwebtoonnaver-webtoon
      pixiv:novelpixiv-novel:novel
      pixiv:novel-userpixiv-novel:user
      pixiv:novel-seriespixiv-novel:series
      pixiv:novel-bookmarkpixiv-novel:bookmark

    • config settings will automatically use the old values

    • target directories using {category} will use the new category names by default

    • use --compat or "category-map": "compat" to restore old category names

  • include exit status bitmasks of NotFoundError (8) and NoExtractorError (64) into general HttpError (4) and InputError (32) respectively

Extractors

Additions

  • [civitai] add search-images and posts extractors (#7609)

  • [comick] add support (#1825 #6782)

  • [dankefuerslesen] add support (#7669)

  • [dynastyscans] add anthology extractor (#7627)

  • [girlsreleased] add support (#6200)

  • [girlswithmuscle] add support (#4493 #6016)

  • [iwara] add support (#2652 #5840 #7785)

  • [kemono] add artists extractor (#7582)

  • [misskey] add avatar, background, and info extractors (#5347)

  • [motherless] add group extractor (#7774 #7787)

  • [naver-chzzk] add comment and community extractors (#7735 #7741)

  • [nudostar] add support (#5735 #6556)

  • [rawkuma] add support (#4571)

  • [redbust] add support (#6759 #6918 #7043)

Fixes

  • [4archive] fix thread extractor

  • [arcalive] fix download URLs (#7678)

  • [arcalive] replace ac-p.namu subdomains with ac-o.namu (#7556)

  • [archivedmoe] fix redirection issue (#7652 #7653 #7664)

  • [aryion] fix favorite extractor (#7775)

  • [batoto] fix downloading manga with alerts/notices (#7657)

  • [behance] fix 403 Forbidden errors (#7710)

  • [bunkr] fix file downloads (#7747)

  • [civitai] fix & update search extractor (#7609)

  • [danbooru] fix Ugoira conversions for posts without ZIP:ZipFileName (#7630)

  • [deviantart:tag] fix username (#7587)

  • [deviantart:tiptap] fix TypeError when textAlign is null (#7639)

  • [directlink] fix config lookups by subcategory (#6582)

  • [discord] support forwarded messages & handle missing threads (#7706 #7722)

  • [furaffinity] fix submissions results (#7759)

  • [hitomi] fix negative tag searches (#7694)

  • [kemono] fix tagged creator posts

  • [mangadex:list] fix config lookups for list-feed subcategory

  • [nijie] fix file extraction (#7624)

  • [paheal] fix 404 Not Found error for tags with URL encoded characters (#7642)

  • [patreon] send Referer header when downloading .m3u8 videos (#7571)

  • [patreon] fix campaign_id extraction from Next.js 13 creator pages (#7773)

  • [readcomiconline] fix extraction (#7606 #7789)

  • [reddit] fix archive IDs of fallback files (#7760)

  • [rule34] fix file downloads (#7697)

  • [sankaku] fix extracting extended tag categories (#7744)

  • [sexcom] prevent .css file downloads (#7632)

  • [skeb] fix KeyError - 'frame_rate' (#7798)

  • [tiktok] handle exceptions when extracting avatars (#7682)

  • [vsco] fix JSON returned by VSCO (#7821)

  • [warosu] HTML attribute fix (#7676 #7677 #7777)

Improvements

  • [artstation] support downloading .mview files (#7812)

  • [civitai] support "My Reactions" results for videos (#7608)

  • [e621] support e621.cc/posts URLs (#6809)

  • [erome] restructure extractor hierarchy (#7804)

  • [everia] prevent redirect when fetching post pages

  • [exhentai] ensure file signature bytes aren`t all zero (#4902)

  • [exhentai] implement "source": "metadata" (#4902)

  • [fanbox] return fileMap files in order (#2718)

  • [gelbooru] improve error message for 401 Unauthorized responses (#7674)

  • [imagevenue] detect 404 image files (#7570)

  • [instagram] provide more descriptive URLs for video_dash_manifest videos (#7631)

  • [pinterest] support pin.it redirects to board (#7805)

  • [pinterest] match board URLs with query strings (#7805)

  • [rule34us] prioritize video.rule34.us for video downloads (#6582)

  • [rule34xyz] implement login with username & password (#7736)

  • [sankaku] allow passing cookies (#7333)

  • [sexcom] support /pics/ URLs (#7611)

  • [tiktok] detect login page redirects (#7716)

  • [vk] detect challenge page redirects (#7650)

  • [vk] prevent 404 Not Found errors for file downloads

  • [vk] add continuation message (#7650)

  • [warosu] detect missing images by checking hostname (#7698 #7699)

  • [ytdl] set domain as subcategory when using Generic extractor (#6582)

Metadata

  • [civitai] always provide file[…] metadata (#7548)

  • [everia] improve filename by unquoting URLs (#7620)

  • [fanbox] extract archives metadata (#7454)

  • [gelbooru_v02] extract total/search_count metadata (#7689)

  • [instagram] provide post_url for stories and highlights (#7810)

  • [kemono:discord] update server & channel metadata (#7569)

  • [mangaread] fix manga_alt metadata

  • [newgrounds] filter <script> content in tags (#7604)

  • [patreon] return metadata for paywalled posts (#6582)

  • [pinterest] remove excess whitespace from description fields (#4335)

  • [pixiv] remove /jump.php from caption links (#4327)

  • [tenor] extract more metadata

  • [twitter] extract source_id and source_user metadata (#7470 #7640)

  • [twitter] extract sensitive_flags metadata (#2523)

  • [vk] fix user metadata extraction

Options

  • [civitai] add option to retrieve post metadata (#7548)

  • [exhentai] add limits-action option (#6504)

  • [fanbox] add fee-max option (#7726)

  • [kemono] extend duplicates option (#7696)

  • [mangadex] allow ratings to be a (comma-separated) string (#7799)

  • [misskey] add include option (#5347)

  • [sankaku] remove id-format option (#5073 #6808)

  • [webtoons] add banners and thumbnails options (#6468 #7441)

Common

  • update browser User-Agents and headers

  • allow using predefined Firefox/Chrome headers & ciphers

  • allow overriding user-agent when browser is used (#7647)

  • support using system certificates via truststore (#6582)

  • fix URLs not getting written to -e/--error-file (#7758)

  • raise ChallengeError for Cloudflare & DDoS-Guard challenge pages (#1945)

  • prevent exceptions for for non-fatal requests (#7598)

  • simplify user extractors by using Dispatch mixin

  • allow GalleryExtractor instances to return additional asset files

Removals

  • [mangasee] remove module

Downloaders

  • support dynamic download rate limits (#7638)

  • [http] fail downloads with HTML content (#4798 #7697)

  • [http] add MIME type and signature check for .html files

  • [http] fix potential FileExistsError when .part file moved (#5385)

  • [http] implement _http_signature checks (#4902)

  • [ytdl] fix KeyError - 'filepath' when using legacy youtube_dl (#6949 #7752 #7824)

  • [ytdl] fix postprocessing/merge errors (#7581)

  • [ytdl] detect yt-dlp independent of module name (#7599)

  • [ytdl] support custom headers when fetching HLS/DASH manifests

Post Processors

  • implement shortcuts for mode and event options (e.g. metadata/jsonl@post)

  • [exec] implement commands option

Options

  • add category-map and config-map (#7612)

  • add signals-actions (#1861 #6582)

  • add --compat command-line option

  • add --Print command-line option

  • swap --print and --Print semantics

Cookies

  • add native support for LibreWolf profiles (#4101 #7625)

  • improve cookie-related logging messages

  • update expired cookie messages (#7644)

Formatter

  • add D conversion - ISO 8601 string to datetime

  • add L conversion - ISO 639-1 code to language name

  • change old L/length conversion to n

  • implement M format specifier

Optimizations

  • replace %-formatted and .format(…) strings with f-strings (#7671)

  • improve regular expression usage

  • replace match.group(N) with match[N] (#7671)

  • update match.lastindex usage

  • remove pre-3.8 workarounds (#7671)

  • remove @staticmethod decorators

  • add request_json() and request_xml() functions

  • slightly improve performance of extract functions

  • slightly improve filter performance

Miscellaneous

  • [actions] add flag and raise actions

  • [job] refactor parent-child config path building (#7527)

  • [job:data] wrap exceptions in a dict (#7723)

  • [path] support character sequences in path-restrict replacements (#1707)

  • [pyinstaller] exclude pkg_resources module (#7592)

  • [scripts/init] add scripts to generate initial extractor code and test results

  • [scripts/options] make output width independent of terminal size

  • fix Last-Modified mtime overwriting post processor mtime (#7529)

  • use pip for make install (#7628)

1.29.7 - 2025-05-23

Extractors

Additions

  • [mangadex] add following extractor (#7487)

  • [pixeldrain] add support for filesystem URLs (#7473)

Fixes

  • [bluesky] handle posts without record data (#7499)

  • [civitai] fix & improve video downloads (#7502)

  • [civitai] fix exception for images without modelVersionId (#7432)

  • [civitai] make metadata extraction non-fatal (#7562)

  • [fanbox] use "browser": "firefox" by default (#7490)

  • [idolcomplex] fix pagination logic (#7549)

  • [idolcomplex] fix 429 error during login by adding a 10s delay

  • [instagram:stories] fix post_date metadata (#7521)

  • [motherless] fix video gallery downloads (#7530)

  • [pinterest] handle story_pin_product_sticker_block blocks (#7563)

  • [subscribestar] fix content and title metadata (#7486 #7526)

Improvements

  • [arcalive] allow overriding default User-Agent header (#7556)

  • [fanbox] update API headers (#7490)

  • [flickr] add info option (#4720 #6817)

  • [flickr] add profile option

  • [instagram:stories] add split option (#7521)

  • [mangadex] implement login with client credentials

  • [mangadex] send Authorization header only when necessary

  • [mastodon] support Akkoma/Pleroma /notice/:ID URLs (#7496)

  • [mastodon] support Akkoma/Pleroma /objects/:UUID URLs (#7497)

  • [pixiv] Implement sanity handling for ugoira works (#4327 #6297 #7285 #7434)

  • [twitter:ctid] reduce chance of generating the same ID

Metadata

  • [civitai] provide proper extension for model files (#7432)

  • [flickr] provide license_name metadata

  • [sankaku] support new tags categories (#7333 #7553)

  • [vipergirls] provide num and count metadata (#7479)

  • [vipergirls] extract more metadata & rename fields (#7479)

Downloaders

  • [http] fix setting mtime per file (#7529)

  • [ytdl] improve temp/part file handling (#6949 #7494)

Cookies

1.29.6 - 2025-05-04

Extractors

Additions

  • [manganelo] support nelomanga.net and mirror domains (#7423)

Fixes

Improvements

  • [kemonoparty] add endpoint option (#7438 #7450 #7462)

  • [tumblr] improve error message for dashboard-only blogs (#7455)

  • [weasyl] support /view/ URLs (#7469)

Metadata

  • [chevereto] extract date metadata (#7437)

  • [civitai] implement retrieving model and version metadata (#7432)

  • [manganelo] extract more metadata

Post Processors

  • [directory] add directory post processor (#7432)

Miscellaneous

  • [job] do not reset skip count when skip-filter fails (#7433)

1.29.5 - 2025-04-26

Extractors

Additions

  • [bluesky] add video extractor (#4438)

  • [instagram] add followers extractor (#7374)

  • [itaku] add stars extractor (#7411)

  • [pictoa] add support (#6683 #7409)

  • [twitter] add followers extractor (#6331)

Fixes

  • [architizer] fix project extractor (#7421)

  • [bluesky:likes] fix infinite loop (#7194 #7287)

  • [deviantart] fix 401 Unauthorized errors for for multi-image posts (#6653)

  • [everia] fix title extraction (#7379)

  • [fanbox] fix comments extraction

  • [fapello] stop pagination on empty results (#7385)

  • [kemonoparty] fix archives option (#7416 #7419)

  • [pixiv] fix user_details requests not being cached (#7414)

  • [pixiv:novel] handle exceptions during embeds extraction (#7422)

  • [subscribestar] fix username & password login

  • [wikifeet] support site redesign (#7286 #7396)

Improvements

  • [bluesky:likes] use repo.listRecords endpoint (#7194 #7287)

  • [gelbooru] don't hardcode image server domains (#7392)

  • [instagram] support /share/ URLs (#7241)

  • [kemonoparty] use /posts-legacy endpoint (#6780 #6931 #7404)

  • [naver] support videos (#4682 #7395)

  • [scrolller] support album posts (#7339)

  • [subscribestar] add warning for missing login cookie

  • [twitter] update API endpoint query hashes (#7382 #7386)

  • [weasyl] use gallery-dl User-Agent header (#7412)

Metadata

  • [deviantart:stash] extract more metadata (#7397)

  • [moebooru:pool] replace underscores in pool names (#4646)

  • [naver] fix recent date bug (#4682)

Post Processors

  • [ugoira] restore keep-files functionality (#7304)

  • [ugoira] support "keep-files": true + custom extension (#7304)

  • [ugoira] use _ugoira_frame_index to detect .zip files

Miscellaneous

  • [util] auto-update Chrome version

  • use internal version of re.compile() for extractor patterns

1.29.4 - 2025-04-13

Extractors

Additions

  • [chevereto] support imagepond.net (#7278)

  • [webtoons] add artist extractor (#7274)

Fixes

  • [deviantart] fix KeyError: 'has_subfolders' (#7272 #7337)

  • [discord] fix parent keyword inconsistency (#7341 #7353)

  • [E621:pool] fix AttributeError (#7265 #7344)

  • [everia] fix/improve image extraction (#7270)

  • [gelbooru] fix video URLs (#7345)

  • [hentai2read] fix AttributeError exception for chapters without artist (#7355)

  • [issuu] fix extractors (#7317)

  • [kemonoparty] fix file paths with backslashes (#7321)

  • [readcomiconline] fix issue extractor (#7269 #7330)

  • [rule34xyz] update to API v2 (#7289)

  • [zerochan] fix KeyError: 'author' (#7282)

Improvements

  • [instagram] use Chrome User-Agent by default (#6379)

  • [pixiv] support phixiv.net URLs (#7352)

  • [tumblr] support URLs without subdomain (#7358)

  • [webtoons] download JPEG files in higher quality

  • [webtoons] use a default 0.5-1.5s delay between requests (#7329)

  • [zzup] support w.zzup.com URLs (#7327)

Downloaders

  • [ytdl] fix KeyError: 'extractor' exception when ytdl reports an error (#7301)

Post Processors

  • [metadata] add metadata-path option (#6582)

  • [metadata] fix handling of empty directory paths (#7296)

  • [ugoira] preserve extension when using "mode": "archive" (#7304)

Miscellaneous

  • [formatter] add i and f conversions (#6582)

1.29.3 - 2025-03-29

Extractors

Additions

  • [danbooru] add favgroup extractor

  • [imhentai] support hentaienvy.com and hentaizap.com (#7192 #7218)

Fixes

  • [bunkr] fix filename extraction (#7237)

  • [deviantart:stash] fix legacy sta.sh links (#7181)

  • [hitomi] fix extractors (#7230)

  • [mangapark] fix extractors (#4999 #5883 #6507 #6908 #7232)

  • [nozomi] fix extractors (#7242)

  • [patreon] include subdomains in session_id cookie check (#7188)

  • [patreon] do not match /messages URLs as creator (#7187)

  • [pinterest] handle story_pin_static_sticker_block blocks (#7251)

  • [sexcom] fix gif pin extraction (#7239)

  • [skeb] make exceptions when extracting posts non-fatal (#7250)

  • [zerochan] parse JSON-LD data (#7178)

Improvements

  • [arcalive] extend gifs option

  • [deviantart] support multiple images for single posts (#6653 #7261)

  • [deviantart] add subfolder support (#4988 #7185 #7220)

  • [deviantart] match /gallery/recommended-for-you URLs (#7168 #7243)

  • [instagram] extract videos from video_dash_manifest data (#6379 #7006)

  • [mangapark] support mirror domains

  • [mangapark] support v3 URLs (#2072)

  • [mastodon] support /statuses URLs (#7255)

  • [sexcom] support new-style /gifs and /videos URLs (#7239)

  • [subscribestar] detect redirects to /age_confirmation_warning pages

  • [tiktok] add retry mechanism to rehydration data extraction (#7191)

Metadata

  • [bbc] extract more metadata (#6582)

  • [kemonoparty] extract archives metadata (#7195)

  • [kemonoparty] enable username/user_profile metadata by default

  • [kemonoparty:discord] always provide channel_name metadata (#7245)

  • [sexcom] extract date_url metadata (#7239)

  • [subscribestar] extract title metadata (#7219)

Downloaders

  • [ytdl] support processing inline HLS/DASH manifest data (#6379 #7006)

Miscellaneous

  • [aes] simplify block_count calculation

  • [common] add subdomains argument to cookies_check() (#7188)

  • [config] fix using the same key multiple times with apply (#7127)

  • [tests] implement expected failures

1.29.2 - 2025-03-15

Extractors

Additions

Fixes

  • [civitai] fix/improve query parameter handling (#7138)

  • [facebook] improve date extraction (#7151)

  • [sankaku] update API URLs (#7154 #7155 #7163)

  • [twitter] prevent exception in _extract_components() (#7139)

Improvements

  • [batoto] add domain option (#7174)

  • [furaffinity] extract scraps metadata (#7015)

  • [tiktok] implement audio extraction without yt-dlp

  • [wikimedia] add subcategories option (#2340)

1.29.1 - 2025-03-08

Extractors

Additions

  • [tenor] add support (#6075)

Fixes

  • [bunkr] update API endpoint (#7097)

  • [erome] fix AttributeError for albums without tags (#7076)

  • [furaffinity] fix artist metadata (#6582 #7115 #7123 #7130)

  • [jpgfish] decrypt file URLs (#7073 #7079 #7109)

  • [sankaku] fix search tag limit check

  • [vsco] fix video extractor (#7113)

  • [vsco] fix extracting videos from /gallery results (#7113)

Improvements

  • [bunkr] add endpoint option (#7097)

  • [danbooru:pool] download posts in pool order, add order-posts option (#7091)

  • [erome:search] recognize all URL query parameters (#7125)

  • [reddit] add selftext option (#7111)

  • [redgifs:search] support /search?query=... URLs (#7118)

  • [sankaku] increase wait time on 429 errors (#7129)

  • [tiktok] improve tiktok-range parsing (#7098)

Downloaders

  • [http] detect Cloudflare/DDoS-Guard challenges (#7066 #7121)

  • warn about invalid subcategory values (#7103 #7119)

1.29.0 - 2025-03-01

Changes

  • build gallery-dl.exe on Windows 10 / Python 3.13 (#6684)

  • provide Windows 7 / Python 3.8 builds as gallery-dl_x86.exe

Extractors

Additions

Fixes

  • [bunkr] decrypt file URLs (#7058 #7070 #7085 #7089 #7090)

  • [chevereto/jpgfish] fix extraction (#7073 #7079)

  • [generic] fix config lookups by subcategory

  • [philomena] fix date values without UTC offset (#6921)

  • [philomena] download full URLs to prevent potential 404 errors (#6922)

  • [pixiv] prevent exceptions during comments extraction (#6965)

  • [reddit] restrict subreddit search results (#7025)

  • [sankaku] fix extraction (#7071 #7072)

  • [subscribestar] fix post extractor (#6582)

  • [twitter] revert generated CSRF token length to 32 characters (#6895)

  • [vipergirls] change default domain to viper.click (#4166)

  • [weebcentral] fix extracting wrong number of chapter pages (#6966)

Improvements

  • [b4k] update domain to arch.b4k.dev (#6955 #6956)

  • [bunkr] update default archive ID format (#6935)

  • [bunkr] provide fallback URLs for 403 download links (#6732 #6972)

  • [bunkr] implement fast --range support (#6985)

  • [furaffinity] use a default delay of 1 second between requests (#7054)

  • [itaku] support gallery section URLs (#6951)

  • [patreon] support /profile/creators URLs

  • [subscribestar] detect and handle redirects (#6916)

  • [twibooru] match URLs with www subdomain (#6903)

  • [twitter] support grok cards content (#7040)

  • [vsco] improve m3u8 handling

  • [weibo] add movies option (#6988)

Metadata

  • [bunkr] extract id_url metadata (#6935)

  • [erome] extract tags metadata (#7076)

  • [issuu] unescape HTML entities

  • [newgrounds] provide comment_html metadata (#7038)

  • [patreon] extract campaign metadata (#6989)

Downloaders

  • implement downloader options per extractor category

  • [http] add sleep-429 option (#6996)

  • [ytdl] support specifying module as filesystem paths (#6991)

Archives

  • [archive] implement support for PostgreSQL databases (#6152)

  • [archive] add archive-table option (#6152)

Miscellaneous

  • [aes] handle errors during cryptodome import (#6906)

  • [executables] fix loading certifi SSL certificates (#6393)

  • improve \f format string handling for --print

1.28.5 - 2025-01-28

Extractors

Additions

Fixes

  • [4archive] fix TypeError

  • [adultempire] bypass age confirmation check

  • [architizer] fix extraction

  • [artstation] avoid Cloudflare challenges (#5817, #5658, #5564, #5554)

  • [deviantart] prevent crash when accessing premium_folder data (#6873)

  • [fapachi] fix extraction (#6881)

  • [issuu] fix user extractor

  • [kemonoparty] fix username metadata and filtering by tag for /posts URLs (#6833)

  • [mangafox] fix chapter extraction

  • [mangahere] fix chapter extraction

  • [pixiv] fix sanity_level workaround (#4327)

  • [pornpics] fix pagination results from HTML pages

  • [twitter] handle exceptions during file extraction (#6647)

  • [vsco] fix JSONDecodeError (#6887, #6891)

  • [weebcentral] fix extraction (#6860)

  • [xhamster] fix gallery extractor (#6818, #6876)

Improvements

  • [batoto] use chapter_id in default archive IDs (#6835)

  • [e621] support e621.cc and e621.anthro.fr frontend URLs (#6809)

  • [e621] prevent premature pagination end (#6886)

  • [facebook] allow accessing all metadata in directory format strings (#6874)

  • [hiperdex] update domain to hiperdex.com

  • [kemonoparty] enable filtering creator posts by tag (#6833)

  • [khinsider] add covers option (#6844)

  • [komikcast] update domain to komikcast.la

  • [lofter] improve error handling (#6865)

  • [pornpics] avoid redirect when retrieving a gallery page

  • [urlgalleries] support new URL format

Metadata

  • [bunkr] extract better filename metadata (#6824)

  • [hiperdex] fix description metadata

  • [khinsider] extract more album metadata (#6844)

  • [mangaread] fix manga metadata extraction

  • [rule34xyz] fix date and tags metadata

  • [saint] fix metadata of /d/ URLs

  • [toyhouse] fix date, artists, and characters metadata

  • [webtoons] fix username and author_name metadata

Removals

  • [cohost] remove module

  • [fanleaks] remove module

  • [shimmie2] remove tentaclerape.net

  • [szurubooru] remove booru.foalcon.com

Miscellaneous

  • [docs] add nix docs to README (#6606)

  • [path] fix exception when using --rename-to + --no-download (#6861)

  • [release] include scripts/run_tests.py in release tarball (#6856)

1.28.4 - 2025-01-12

Extractors

Additions

Fixes

  • [bunkr] update to new site layout (#6798, #6805)

  • [bunkr] fix ValueError on relative redirects (#6790)

  • [plurk] fix user data extraction and make it non-fatal (#6742)

Improvements

  • [bunkr] support /f/ media URLs

  • [e621] accept tag search URLs with empty tag (#6783)

  • [pixiv] provide fallback URLs (#6762)

  • [wallhaven] extract search[tags] and search[tag_id] metadata (#6772)

Miscellaneous

  • [util] support not splitting value argument when calling contains() (#6773)

1.28.3 - 2025-01-04

Extractors

Additions

  • [civitai] add user-videos extractor (#6644)

  • [szurubooru] support visuabusters.com/booru (#6729)

Fixes

  • [8muses] skip albums without valid permalink (#6717)

  • [batoto] update domains (#6714)

  • [deviantart:tiptap] fix deviation embeds without token

  • [hitomi] fix searches (#6713)

  • [instagram:reels] fix pinned values (#6719)

  • [kemonoparty] handle discord favorites (#6706)

  • [piczel] fix extraction (#6735)

  • [poipiku] fix downloads when post has a warning (#6736)

  • [sankaku] support alphanumeric book/pool IDs (#6757)

  • [subscribestar] fix attachment downloads (#6721, #6724, #6758)

  • [subscribestar] improve content metadata extraction (#6761)

  • [tapas] fix TypeError for locked episodes (#6700)

Improvements

  • [boosty] support file post attachments (#6760)

  • [deviantart:tiptap] support more content block types (#6686)

  • [directlink] use domain as subcategory (#6703)

  • [hitomi] provide search_tags metadata for tag and search results (#6756)

  • [subscribestar] support audio files (#6758)

Miscellaneous

  • [workflows:executables] build with Python 3.13

1.28.2 - 2024-12-20

Extractors

Additions

Fixes

  • [facebook] decode Unicode surrogate pairs in metadata values (#6599)

  • [zerochan] parse API responses manually when receiving invalid JSON (#6632)

  • [zerochan] fix source metadata extraction when not logged in

Improvements

  • [bilibili] extract files from module_top entries (#6687)

  • [bilibili] support /upload/opus URLs (#6687)

  • [bluesky] default to posts timeline when reposts or quoted is enabled (#6583)

  • [common] simplify HTTP error messages

  • [common] detect DDoS-Guard challenge pages

  • [deviantart] improve tiptap markup to HTML conversion (#6686)

    • fix KeyError: 'attrs' for links without href

    • support heading content blocks

    • support strike text markers

  • [instagram] extract date metadata for stories (#6677)

  • [kemonoparty:favorite] support new URL format (#6676)

  • [saint] support saint2.cr URLs (#6692)

  • [tapas] improve extractor hierarchy (#6680)

Options

  • [cohost] add avatar and background options (#6656)

Miscellaneous

  • support * wildcards for parent>child categories, for example reddit>* (#6673)

  • use latest Firefox UA as default user-agent

  • use random unused port for "user-agent": "browser" requests

1.28.1 - 2024-12-07

Extractors

Additions

  • [bluesky] add info extractor

Fixes

  • [bluesky] fix exception when encountering non-quote embeds (#6577)

  • [bluesky] unescape search queries (#6579)

  • [common] restore using environment proxy settings by default (#6553, #6609)

  • [common] improve handling of user-agent settings (#6594)

  • [e621] fix TypeError when metadata is enabled (#6587)

  • [gofile] fix website token extraction (#6596)

  • [inkbunny] fix re-login loop (#6618)

  • [instagram] handle empty carousel_media entries (#6595)

  • [kemonoparty] fix o query parameter handling (#6597)

  • [nhentai] fix download URLs (#6620)

  • [readcomiconline] fix chapter extraction (#6070, #6335)

  • [realbooru] fix extraction (#6543)

  • [rule34] fix favorite extraction (#6573)

  • [zerochan] download .webp and .gif files (#6576)

Improvements

  • [hentaicosplays] update domains (#6578)

  • [pixiv:ranking] implement filtering results by content (#6574)

  • [pixiv] include user ID in failed AJAX request warnings (#6581)

Options

  • [patreon] add format-images option (#6569)

  • [zerochan] add extensions option (#6576)

1.28.0 - 2024-11-30

Changes

  • [common] disable using environment network settings by default (HTTP_PROXY, .netrc, …)

    • disable trust_env session attribute

    • disable Authorization header injection from .netrc auth (#5780, #6134, #6455)

    • add proxy-env option

  • [ytdl] change forward-cookies default value to true (#6401, #6348)

Extractors

Additions

Fixes

  • [8chan] avoid performing network requests within _init() (#6387)

  • [bluesky] fix downloads from non-bsky PDSs (#6406)

  • [bunkr] fix album names containing <>& characters

  • [flickr] use download URLs (#6360, #6464)

  • [hiperdex] update domain to hipertoon.com (#6420)

  • [imagechest] fix extractors (#6475, #6491)

  • [instagram] fix using numeric cursor values (#6414)

  • [kemonoparty] update to new site layout (#6415, #6503, #6528, #6530, #6536, #6542, #6554)

  • [koharu] update domain to niyaniya.moe (#6430, #6432)

  • [mangadex] apply lang option only to chapter results (#6372)

  • [newgrounds] fix metadata extraction (#6463, #6533)

  • [nhentai] support .webp files (#6442, #6479)

  • [patreon] use legacy mobile UA when no session_id is set

  • [pinterest] update API headers (#6513)

  • [pinterest] detect video/audio by block content (#6421)

  • [scrolller] prevent exception for posts without mediaSources (#5051)

  • [tumblrgallery] fix file downloads (#6391)

  • [twitter] make source metadata extraction non-fatal (#6472)

  • [weibo] fix livephoto filename & extension (#6471)

Improvements

  • [bluesky] support main.bsky.dev URLs (#4438)

  • [bluesky] match common embed fixes (#6410, #6411)

  • [boosty] update default video format list (#2387)

  • [bunkr] support bunkr.cr URLs

  • [common] allow passing cookies to OAuth extractors

  • [common] allow overriding more default User-Agent headers (#6496)

  • [philomena] switch default ponybooru filter (#6437)

  • [pinterest] support story_pin_music blocks (#6421)

  • [pixiv] get ugoira frame extension from meta_single_page values (#6056)

  • [reddit] support user profile share links (#6389)

  • [steamgriddb] disable adjust-extensions for fake-png files (#5274)

  • [twitter] remove cookies migration workaround

Metadata

  • [bluesky] provide author[instance] metadata (#4438)

  • [instagram] fix extension of apparent .webp files (#6541)

  • [pillowfort] provide count metadata (#6478)

  • [pixiv:ranking] add rank metadata field (#6531)

  • [poipiku] return count as proper number (#6445)

  • [webtoons] extract episode_no for comic results (#6439)

Options

  • [civitai] add metadata option - support fetching generation data (#6383)

  • [exhentai] implement tags option (#2117)

  • [koharu] implement tags option

  • [rule34xyz] add format option (#1078)

Downloaders

  • [ytdl] fix AttributeError caused by decodeOption() removal (#6552)

Post Processors

  • [classify] rewrite - fix skipping existing files (#5213)

  • enable inheriting options from global postprocessor objects

  • allow postprocessors values to be a single post processor object

Cookies

  • support Chromium table version 24 (#6162)

  • fix GCM pad length calculation (#6162)

  • try decryption with empty password as fallback

Documentation

  • update recommended pip command for installing dev version (#6493)

  • update gallery-dl.conf (#6501)

Options

  • add -4/--force-ipv4 and -6/--force-ipv6 command-line options

  • fix passing negative numbers as arguments (#5262)

Miscellaneous

  • [output] use default ANSI colors only when stream is a TTY

  • [util] implement defaultdict filters-environment

  • [util] enable using multiple statements for all filter options (#5262)

1.27.7 - 2024-10-25

Extractors

Additions

  • [civitai] add extractors for global models and images (#6310)

  • [mangadex] add author extractor (#6372)

  • [scrolller] add support (#295, #3418, #5051)

Fixes

  • [8chan] automatically detect TOS cookie name (#6318)

  • [bunkr] update to new site layout (#6344, #6352, #6368)

  • [bunkr] send proper Referer headers for file downloads (#6319)

  • [civitai] add uuid metadata field & use it as default archive format (#6326)

  • [civitai] fix "My Reactions" results (#6263)

  • [civitai] fix model file download URLs for tRPC API

  • [lensdump] fix extraction (#6313)

  • [pixiv] make retrieving ugoira metadata non-fatal (#6297)

  • [pixiv] fix exception when processing deleted sanity_level works (#6339)

  • [urlgalleries] fix extraction

  • [wikimedia] fix non-English Fandom/wiki.gg articles (#6370)

Improvements

  • [8chan] support /last/ thread URLs (#6318)

  • [bunkr] support bunkr.ph and bunkr.ps URLs

  • [newgrounds] support page numbers in URLs (#6320)

  • [patreon] support /c/ prefix in creator URLs (#6348)

  • [pinterest] support story pins (#6188, #6078, #4229)

  • [pixiv] implement sanity_level workaround for user artworks results (#4327, #5435, #6339)

Options

  • [bluesky] add quoted option (#6323)

  • [pixiv] add captions option (#4327)

  • [reddit] add embeds option (#6357)

  • [vk] add offset option (#6328)

Downloaders

  • [ytdl] implement explicit HLS/DASH handling

Post Processors

  • add error event

Miscellaneous

  • [cookies] convert Chromium expires_utc values to Unix timestamps

  • [util] add std object to global eval namespace (#6330)

  • add --print and --print-to-file command-line options (#6343)

  • use child extractor fallbacks only when a non-user error occurs (#6329)

1.27.6 - 2024-10-11

Extractors

Additions

  • [ao3] add subscriptions extractor (#6247)

  • [boosty] add support (#2387)

  • [civitai] add post extractors (#6279)

  • [pixiv] support unlisted artworks (#5162)

Fixes

  • [cohost] sanitize default filenames (#6262)

    • limit headline length

    • remove plainTextBody

  • [deviantart] fix & improve journal/literature extraction (#6254, #6207, #6196)

    • extract journal HTML from webpage if possible

    • support converting tiptap markup to HTML

  • [deviantart] fix stash folder extraction

  • [flickr] update default API credentials (#6300)

  • [flickr] fix ZeroDivisionError (#6252)

  • [imagefap] fix {num} in single image default filenames

  • [myhentaigallery] fix tags extraction

  • [patreon] extract attachments_media files (#6241, #6268)

  • [pixiv] implement workaround for limit_sanity_level works (#4327, #4747, #5054, #5435, #5651, #5655)

  • [pornhub] fix KeyError when album images are missing (#6299)

  • [rule34us] fix extraction (#6289)

  • [8chan] set TOS cookie for current and previous day

Improvements

  • [bunkr] support bunkr.pk URLs (#6272)

  • [civitai] use tRPC API by default (#6279)

  • [civitai] improve default archive format (#6302)

  • [komikcast] update domain to komikcast.cz

  • [newgrounds] detect more comment embeds (#6253)

  • [newgrounds] add more fallback URL formats for art-images files

  • [oauth] prevent empty browser names

  • [patreon] use mobile UA (#6241, #6239, #6140)

  • [patreon] handle suspended accounts

  • [pixiv] detect works requiring My pixiv access

Metadata

  • [civitai] ensure image files have an id (#6251)

  • [gelbooru_v02] unescape HTML entities in categorized tags

  • [generic] ensure path metadata is always defined

  • [pixiv] retrieve caption from AJAX API when empty (#4327, #5191)

Options

  • [fanbox] add comments option, extend metadata option (#6287)

  • [pixiv] add comments option (#6287)

Removals

  • [blogger] remove micmicidol.club

  • [chevereto] remove deltaporno.com

  • [lolisafe] remove xbunkr.com

  • [pururin] remove module

  • [shimmie2] remove loudbooru.com

Post Processors

  • [ugoira] fix BadZipFile exceptions (#6285)

  • [ugoira] catch all exceptions when extracting ZIP archives (#6285)

  • [ugoira] forward frame data as _ugoira_frame_data (#6154, #6285)

Miscellaneous

  • [build] remove setuptools and requests version restrictions

  • [docker] build from python:3.12-alpine

  • [text] improve parse_query() performance

1.27.5 - 2024-09-28

Extractors

Additions

Fixes

  • [8chan] update TOS cookie name

  • [deviantart] work around OAuth API returning empty journal texts (#6196, #6207, #5916)

  • [weasyl:favorite] fix pagination (#6113)

Improvements

  • [bluesky] support video downloads (#6183)

  • [deviantart] add previews option (#3782, #6124)

  • [deviantart] warn about empty journal texts (#5916)

  • [inkbunny:favorite] update default directory (#6115)

  • [jpgfish] update domain to jpg5.su (#6231)

  • [skeb] prevent 429 errors and need for request_key cookie

  • [weasyl:favorite] support readable URL format (#6113)

  • [wikimedia] automatically detect API endpoint when none is defined

  • [zzup] support up.zzup.com galleries (#6181)

Post Processors

  • [ugoira] implement storing "original" frames in ZIP archives (#6147)

  • [ugoira] fix KeyError: '_ugoira_frame_index' (#6154)

Formatter

  • add L conversion - returns the length of a value

  • allow accessing util.NONE via global _nul

Miscellaneous

  • [cookies] add cookies-select option

  • [cookies:firefox] support using domain & container filters together

  • [docker] prevent errors in Dockerfile build

  • [tests] make #category result entries optional

  • allow filtering --list-extractors results

  • implement alternatives for deprecated utc datetime functions

1.27.4 - 2024-09-06

Extractors

Additions

  • [sexcom] add likes extractor (#6149)

  • [wikimedia] add wiki extractor (#6050)

Fixes

  • [bunkr] fix file downloads (#6037)

  • [cyberdrop] fix extraction

  • [deviantart] fix "pagination": "manual" for cursor-based endpoints (#6079)

  • [deviantart] fix "original": "images" (#6124)

  • [exhentai] fix limits option (#6090)

  • [flickr] make exif and context metadata extraction non-fatal (#6002, #6077)

  • [flickr] make album metadata extraction non-fatal (#3441)

  • [furaffinity] fix favorite pagination (#6151)

  • [gofile] fix KeyError: 'childrenIds' (#5993)

  • [newgrounds] fix warning for age-restricted posts (#6005)

  • [toyhouse] fix extraction of image URLs

  • [tumblr] fix 401 Unauthorized for likes when using api-key (#5994)

  • [twitter] fix pinned Tweet extraction (#6102)

  • [ytdl] fix processing playlists of playlists (#6127)

Improvements

  • [bcbnsfw] use * as query when retrieving all posts (#6135)

  • [bunkr] support bunkr: URL prefix (#6017)

  • [e621] cache pool metadata API calls (#6001)

  • [generic] better directory names (#6104)

  • [koharu] improve format selection (#6088)

  • [pixiv] implement downloading "original" ugoira frames (#6056)

  • [pixiv] use mobile API for series (#5983)

Metadata

  • [batoto] improve chapter info regex (#5988, #5997)

  • [batoto] extract chapter_url metadata (#5562)

  • [batoto] improve title extraction (#5988)

  • [hitomi] extract extension_original metadata (#6049)

  • [instagram] add post_date metadata field (#6081, #6091)

  • [sankaku] restore old tags format (#6043)

  • [twitter] extract type metadata (#6111)

Options

  • [bunkr] add tlds option to match URLs with all possible TLDs (#5875, #6017)

  • [instagram] add max-posts option (#6054)

  • [instagram] add info as a possible include value

  • [instagram] allow disabling cursor output

  • [twitter] add info as a possible include value (#6114)

  • [twitter] allow disabling cursor output (#5990)

Post Processors

  • [hash] add hash post processor to compute file hash metadata (#6099)

  • [metadata] add include and exclude options (#6058)

  • [metadata] fix using .. in directories on Windows (#5942, #6094)

  • [rename] add rename post processor to rename previously downloaded files (#5846, #6044)

  • [ugoira] support converting "original" frames (#6056)

  • [ugoira] add skip option (#6056)

Miscellaneous

  • [cookies:firefox] extract only cookies without container by default (#5957)

  • [formatter] implement A format specifier (#6036)

  • [tests] fix bug when running tests in a certain order

  • [util] extend CustomNone with arithmetic operators (#6007, #6009)

  • add --rename and --rename-to command-line options (#5846, #6044)

  • add input-files config option (#6059)

1.27.3 - 2024-08-10

Extractors

Additions

  • [bunkr] support bunkr.ci and bunkrrr.org (#5970)

  • [furaffinity] add submissions extractor (#5954)

  • [hentaicosplays] support hentai-cosplay-xxx.com (#5959)

Fixes

  • [behance] fix KeyError: 'fields' (#5965)

  • [behance] fix video extraction (#5965)

  • [cien] extract all files when authenticated (#5934)

  • [deviantart] fix KeyError - 'category' (#5960, #5961, #5969, #5971, #5976, #5978)

  • [fanbox] update pagination logic (#5949, #5951, #5956)

  • [hotleak] fix AttributeError (#5950)

  • [instagram] restore GraphQL API functionality (#5920)

  • [twitter] update x-csrf-token header during login (#5945)

Improvements

  • [bunkr] fail downloads for maintenance files (#5952)

  • [zerochan] improve tag redirect handling, add redirects option (#5891)

Post Processors

  • [metadata] add base-directory option (#5262, #5728)

1.27.2 - 2024-08-03

Extractors

Additions

  • [agnph] add tag and post extractors (#5284, #5890)

  • [aryion] add favorite extractor (#4511, #5870)

  • [cien] add support (#2885, #4103, #5240)

  • [instagram] add info extractor (#5262)

  • [koharu] add gallery, search, and favorite extractors (#5893, #4707)

  • [twitter] add info extractor (#3623)

Fixes

  • [8chan] update TOS cookie name (#5868)

  • [behance] fix image extraction (#5873, #5926)

  • [booru] prevent crash when file URL is empty (#5859)

  • [deviantart] try to work around journal/status API changes (#5916)

  • [hentainexus] fix error with spread pages (#5827)

  • [hotleak] fix faulty image URLs (#5915)

  • [inkbunny:following] fix potentially infinite loop

  • [nijie] fix image URLs of single image posts (#5842)

  • [readcomiconline] fix extraction (#5866)

  • [toyhouse] fix Content Warning bypass (#5820)

  • [tumblr] revert to offset pagination, implement pagination option (#5880)

  • [twitter] fix username-alt option name (#5715)

  • [warosu] fix extraction

  • [zerochan] handle KeyError - 'items' (#5826)

  • [zerochan] fix error on tag redirections (#5891)

  • [zerochan] fix Invalid control character errors (#5892)

Improvements

  • [bunkr] support bunkr.fi domain (#5872)

  • [deviantart:following] use OAuth API endpoint (#2511)

  • [directlink] extend recognized file extensions (#5924)

  • [exhentai] improve error message when temporarily banned (#5845)

  • [gelbooru_v02] use total number of posts as pagination end marker (#5830)

  • [imagefap] add enumeration index to default filenames (#1746, #5887)

  • [paheal] implement fast --range support (#5905)

  • [redgifs] support URLs with numeric IDs (#5898, #5899)

  • [sankaku] match URLs with www subdomain (#5907)

  • [sankakucomplex] update domain to news.sankakucomplex.com

  • [twitter] implement cursor support (#5753)

  • [vipergirls] improve thread URL pattern

  • [wallpapercave] support album listings (#5925)

Metadata

  • [dynastyscans] extract chapter tags (#5904)

  • [erome] extract date metadata (#5796)

  • [furaffinity] extract folders and thumbnail metadata (#1284, #5824)

  • [sankaku] implement notes extraction (#5865)

  • [subscribestar] fix date parsing in updated posts (#5783)

  • [twitter] extract bookmark_count and view_count metadata (#5802)

  • [zerochan] fix source metadata

  • [zerochan] fix tag category extraction (#5874)

  • [zerochan] delay fetching extended metadata (#5869)

Options

  • [agnph] implement tags option (#5284)

  • [booru] allow multiple url keys (#5859)

  • [cien] add files option (#2885)

  • [koharu] add cbz and format options (#5893)

  • [vsco] add include option (#5911)

  • [zerochan] implement tags option (#5874)

Removals

  • [fallenangels] remove module

Post Processors

  • [metadata] allow using format strings for directory (#5728)

Options

  • add --print-traffic command-line option

  • add -J/--resolve-json command-line option (#5864)

  • add filters-environment option

  • implement archive-event option (#5784)

Actions

  • [actions] support multiple actions per pattern

  • [actions] add exec action (#5619)

  • [actions] add abort and terminate actions (#5778)

  • [actions] allow setting a duration for wait

  • [actions] emit logging messages before waiting/exiting/etc

Tests

  • [tests] enable test results for external extractors (#5262)

  • [tests] load results from ${GDL_TEST_RESULTS} (#5262)

Miscellaneous

  • [cookies] add thorium support (#5781)

  • [job] add resolve argument to DataJob (#5864)

  • [path] fix moving temporary files across drives on Windows (#5807)

  • [ytdl] fix --cookies-from-browser option parsing (#5885)

  • make exceptions in filters/conditionals non-fatal

  • update default User-Agent header to Firefox 128 ESR

  • include zstd in Accept-Encoding header when supported

1.27.1 - 2024-06-22

Extractors

Additions

  • [hentainexus] restore module (#5275, #5712)

  • [shimmie2] support vidya.pics (#5632)

  • [tcbscans] support other domains (#5774)

Fixes

  • [deviantart] fix watching module ID extraction (#5696, #5772)

  • [fanbox] handle KeyError for no longer existing plans (#5759)

  • [kemonoparty:favorite] fix exception when sorting null objects (#5692. #5721)

  • [skeb] fix 429 Too Many Requests errors (#5766)

  • [speakerdeck] fix extraction (#5730)

  • [twitter] fix duplicate ArkoseLogin check

Improvements

  • [nijie] support downloading videos (#5707, #5617)

  • [philomena] support downloading .svg files (#5643)

  • [szurubooru] support empty tag searches (#5711)

  • [twitter] ignore Unavailable media (#5736)

Metadata

  • [hitomi] extract title_jpn metadata (#5706)

  • [instagram] extract liked metadata (#5609)

Options

  • [newgrounds] extend format option (#5709)

  • [twitter] extend ratelimit option (#5532)

  • [twitter] add username-alt option (#5715)

Removals

  • [photobucket] remove module

  • [nitter] remove instances

  • [vichan] remove wikieat.club

Downloaders

  • [ytdl] fix exception due to missing ext in unavailable videos (#5675)

Formatter

  • implement C format specifier (#5647)

  • implement X format specifier (#5770)

Options

  • add --no-input command-line and input config option (#5733)

  • add --config-open command-line option (#5713)

  • add --config-status command-line option (#5713)

Miscellaneous

  • [actions] fix exception when msg is not a string (#5683)

1.27.0 - 2024-06-01

Extractors

Additions

  • [mastodon] add favorite, list, and hashtag extractors (#5529)

  • [mastodon] add support for card images

  • [pixeldrain] add support for single-file album downloads (#5641)

  • [reddit] support comment embeds (#5366)

  • [seiga] re-implement login with username & password

  • [tapas] add creator extractor (#5306)

  • [vsco] add avatar extractor (#5341)

  • [wikimedia] support wiki.gg wikis

Fixes

  • [4archive] fix extraction

  • [8chan] fix file downloads by sending a TOS cookie (#5578)

  • [artstation] disable TLS 1.2 ciphers by default (#5564, #5658)

  • [bluesky] filter reposts only for user timelines (#5528)

  • [common] disable check_hostname for custom SSLContexts (#3614, #4891, #5576)

  • [exhentai] fix Multi-Page Viewer detection (#4969)

  • [exhentai] fix blank page detection

  • [hiperdex] update domain to hiperdex.top (#5635)

  • [hotleak] download files returning a 404 status code (#5395)

  • [imgur] match URLs with title slugs (#5593)

  • [kemonoparty] fix KeyError - 'path' for posts without files (#5368, #5394, #5422, #5488)

  • [kemonoparty] fix crash on posts with missing datetime info (#5422)

  • [mastodon] send canonical true/false boolean values (#5516)

  • [newgrounds] update and fix login procedure (#5109)

  • [patreon] fix bootstrap data extraction (#5624)

  • [poipiku] fix downloading R-18 posts (#5567)

  • [poipoku] avoid language-specific extraction (#5590, #5591)

  • [realbooru] fix videos and provide fallback URLs (#2530)

  • [slideshare] fix extraction

  • [subscribestar] fix file URLs (#5631)

  • [twitter] update domain to x.com (#5597)

  • [twitter] transfer twitter.com cookies to x.com (#5597)

  • [twitter] prevent crash when extracting birdwatch metadata (#5403)

  • [twitter] handle missing expanded_url fields (#5463, #5490)

  • [wikimedia] suppress exception for entries without imageinfo (#5384)

  • [wikimedia] fix exception for files with empty metadata

Improvements

  • [exhentai] detect CAPTCHAs during login (#5492)

  • [foolfuuka] improve board pattern & support pages (#5408)

  • [furaffinity] match fxfuraffinity.net/fxraffinity.net/xfuraffinity.net URLs (#5511, #5568)

  • [gelbooru] improve pagination logic for meta tags (#5478)

  • [kemonoparty:favorite] return artists/posts in native order and support sort and order query parameters (#5375, #5620)

  • [oauth] use Extractor.request() for HTTP requests to support proxy servers etc (#5433)

  • [pixiv] change sanity_level debug message to a warning (#5180)

  • [twitter] improve username & password login procedure (#5445)

  • [twitter] wait for rate limit reset before encountering a 429 error (#5532)

  • [twitter] match fixvx.com URLs (#5511)

  • [twitter] match Tweet URLs with query parameters (#5371, #5372)

  • [twitter] match /photo/ and /video/ Tweet URLs (#5443, #5601)

Options

  • [common] add sleep-429 option (#5160)

  • [common] implement skip-filter option (#5255)

  • [common] implement keywords-eval option (#5621)

  • [kemonoparty] add announcements option (#5262)

  • [pixiv:novel] add covers option (#5373)

  • [twitter] implement relogin option (#5445)

Downloaders

  • [http] add MIME type and signature for .m4v files (#5505)

Post Processors

  • [mtime] do not overwrite _mtime values with None (#5439)

  • [ugoira] log errors for general exceptions

Archives

  • [archive] move DownloadArchive code into its own module

  • [archive] implement DownloadArchiveMemory class (#5255)

  • [archive] add archive-mode option (#5255)

Cookies

  • [cookies] use temporary file when saving cookies.txt files (#5461)

  • [cookies] optimize _find_most_recently_used_file() for exact profiles (#5538)

  • [cookies] set proper expires value for Chrome session cookies

Documentation

  • [docs] update docs/configuration links (#5059, #5369, #5423)

  • [docs] update link to "nightly" builds (#5618)

  • [docs] replace AnchorJS with custom script

  • [docs] update defaults of sleep-request, browser, tls12

  • [docs] complete Authentication info in docs/supportedsites

Formatter

  • [formatter] allow dots in '...' literals (#5539)

Output

  • [output] enable colored output by default

  • [output] extend output.colors (#2566)

  • [output] support NO_COLOR environment variable

  • [output] add --no-colors command-line option

  • [output] add -w/--warning command-line option (#5474)

Tests

  • [tests] select unused port number for local HTTP server

  • [tests] allow filtering extractor result tests by URL or comment

  • [tests] mark tests with missing auth as only_matching

Update

  • implement update-related command-line options (#5233)

    • -U/--update updates an executable file to the latest release

    • --update-check checks if the local version is up to date

    • --update-to allows switching to a different release channel (stable or dev) as well as upgrading/downgrading to a specific tag.

    • (non-executable installations have only access to -U/--update-check for version checks)

Miscellaneous

  • add workaround for requests 2.32.3 issues (#5665)

  • fix exit status of --clear-cache/--list-extractors/--list-modules

  • restore LD_LIBRARY_PATH for executables built with PyInstaller (#5421)

  • store match and groups values in Extractor objects

1.26.9 - 2024-03-23

Extractors

Additions

  • [artstation] support video clips (#2566, #3309, #3911)

  • [artstation] support collections (#146)

  • [deviantart] recognize deviantart.com/stash/… URLs

  • [idolcomplex] support new pool URLs

  • [lensdump] recognize direct image links (#5293)

  • [skeb] add extractor for followed users (#5290)

  • [twitter] add quotes extractor (#5262)

  • [wikimedia] support azurlane.koumakan.jp (#5256)

  • [xvideos] support /channels/ URLs (#5244)

Fixes

  • [artstation] fix handling usernames with dashes in domain names (#5224)

  • [bluesky] fix not spawning child extractors for followed users (#5246)

  • [deviantart] handle CloudFront blocks (#5363)

  • [deviantart:avatar] fix index for URLs without ? (#5276)

  • [deviantart:stash] fix index values (#5335)

  • [gofile] fix extraction

  • [hiperdex] update URL patterns & fix manga metadata (#5340)

  • [idolcomplex] fix metadata extraction

  • [imagefap] fix folder extraction (#5333)

  • [instagram] make accessing like_count non-fatal (#5218)

  • [mastodon] fix handling null moved account field (#5321)

  • [naver] fix EUC-KR encoding issue in old image URLs (#5126)

  • [nijie] increase default delay between requests (#5221)

  • [nitter] ignore invalid Tweets (#5253)

  • [pixiv:novel] fix text extraction (#5285, #5309)

  • [skeb] retry 429 responses containing a request_key cookie (#5210)

  • [warosu] fix crash for threads with deleted posts (#5289)

  • [weibo] fix retweets (#2825, #3874, #5263)

  • [weibo] fix livephoto filename extensions (#5287)

  • [xvideos] fix galleries with more than 500 images (#5244)

Improvements

  • [bluesky] improve API error messages

  • [bluesky] handle posts with different embed structure

  • [deviantart:avatar] ignore default avatars (#5276)

  • [fapello] download full-sized images (#5349)

  • [gelbooru:favorite] automatically detect returned post order (#5220)

  • [imgur] fail downloads when redirected to removed.png (#5308)

  • [instagram] raise proper error for missing reels_media (#5257)

  • [instagram] change posts are private exception to a warning (#5322)

  • [reddit] improve preview fallback formats (#5296, #5315)

  • [steamgriddb] raise exception for deleted assets

  • [twitter] handle "account is temporarily locked" errors (#5300)

  • [weibo] rework pagination logic (#4168)

  • [zerochan] fetch more posts by using the API (#3669)

Metadata

  • [bluesky] add instance metadata field (#4438)

  • [gelbooru:favorite] add date_favorited metadata field

  • [imagefap] extract folder metadata (#5270)

  • [instagram] default likes to 0 (#5323)

  • [kemonoparty] add revision_count metadata field (#5334)

  • [naver] unescape post title and description

  • [pornhub:gif] extract viewkey and timestamp metadata (#4463)

  • [redgifs] make date available for directories (#5262)

  • [subscribestar] fix date metadata

  • [twitter] add birdwatch metadata field (#5317)

  • [twitter] add protected metadata field (#5327)

  • [warosu] fix board_name metadata

Options

  • [bluesky] add reposts option (#4438, #5248)

  • [deviantart] add comments-avatars option (#4995)

  • [deviantart] extend metadata option (#5175)

  • [flickr] add contexts option (#5324)

  • [gelbooru:favorite] add order-posts option (#5220)

  • [kemonoparty] add order-revisions option (#5334)

  • [vipergirls] add like option (#4166)

  • [vipergirls] add domain option (#4166)

Downloaders

  • [http] add MIME type and signature for .mov files (#5287)

Docker

  • build images from source instead of PyPI package

  • build linux/arm64 images (#5227)

  • build images on every push to master

    • tag images as YYYY.MM.DD

    • tag the most recent build from master as dev

    • tag the most recent release build as latest

  • reduce image size (#5097)

Miscellaneous

  • [formatter] fix local DST datetime offsets for :O

  • build Linux executable on Ubuntu 22.04 LTS (#4184)

  • automatically create directories for logging files (#5249)

1.26.8 - 2024-02-17

Extractors

Additions

  • [bluesky] add support (#4438, #4708, #4722, #5047)

  • [bunkr] support new domains (#5114, #5130, #5134)

  • [fanbox] add home and supporting extractors (#5138)

  • [imagechest] add user extractor (#5143)

  • [imagetwist] add gallery extractor (#5190)

  • [kemonoparty] add posts extractor (#5194, #5198)

  • [twitter] support communities (#4913)

  • [vsco] support spaces (#5202)

  • [weibo] add gifs option (#5183)

  • [wikimedia] support www.pidgi.net (#5205)

  • [wikimedia] support bulbapedia.bulbagarden.net (#5206)

Fixes

  • [archivedmoe] fix thebarchive WebM URLs (#5116)

  • [batoto] fix crash when manga name or chapter contains a - (#5200)

  • [bunkr] fix extraction (#5088, #5151, #5153)

  • [gofile] update website_token extraction

  • [idolcomplex] fix pagination for tags containing : (#5184)

  • [kemonoparty] fix deleting file names when computing revision_hash (#5103)

  • [luscious] fix IndexError for files without thumbnail (#5122, #5124, #5182)

  • [naverwebtoon] fix title for comics with empty tags (#5120)

  • [pinterest] fix section URLs for boards with /, ?, or # in their name (#5104)

  • [twitter] update query hashes

  • [zerochan] fix skipping every other post

Improvements

  • [deviantart] skip locked/blurred posts (#4567, #5193)

  • [deviantart] implement downloading PNG versions of non-original images with "quality": "png" (#4846)

  • [flickr] handle non-JSON errors (#5131)

  • [idolcomplex] support alphanumeric post IDs (#5171)

  • [kemonoparty] implement filtering duplicate revisions with "revisions": "unique"(#5013)

  • [naverwebtoon] support /webtoon/ paths for all comics (#5123)

Metadata

  • [idolcomplex] extract id_alnum metadata (#5171)

  • [pornpics] support multiple values for channel (#5195)

  • [sankaku] add id-format option (#5073)

  • [skeb] add num and count metadata fields (#5187)

Downloaders

Fixes

  • [http] remove pyopenssl import (#5156)

Miscellaneous

  • fix filename formatting silently failing under certain circumstances (#5185, #5186)

1.26.7 - 2024-01-21

Extractors

Additions

Fixes

  • [blogger] fix lh-*.googleusercontent.com URLs (#5091)

  • [bunkr] update domain (#5088)

  • [deviantart] fix AttributeError for URLs without username (#5065)

  • [deviantart] fix KeyError: 'premium_folder_data' (#5063)

  • [deviantart:avatar] fix exception when comments are enabled (#4995)

  • [fuskator] make metadata extraction non-fatal (#5039)

  • [gelbooru] only log "Incomplete API response" for favorites (#5045)

  • [giantessbooru] update domain

  • [issuu] fix extraction

  • [nijie] fix download URLs of single image posts (#5049)

  • [patreon] fix KeyError: 'name' (#5048, #5069, #5093)

  • [pixiv] update API headers (#5029)

  • [realbooru] fix download URLs of older posts

  • [twitter] revert to using media timeline by default (#4953)

  • [vk] transform image URLs to non-blurred versions (#5017)

Improvements

  • [batoto] support more mirror domains (#5042)

  • [batoto] improve v2 manga URL pattern

  • [gelbooru] support all tag and URLs with empty tags (#5076)

  • [patreon] download m3u8 manifests with ytdl

  • [sankaku] support post URLs with alphanumeric IDs (#5073)

Metadata

  • [batoto] improve manga_id extraction (#5042)

  • [erome] fix count metadata

  • [kemonoparty] add revision_hash metadata (#4706, #4727, #5013)

  • [paheal] fix source metadata

  • [webtoons] extract more metadata (#5061, #5094)

Removals

  • [chevereto] remove pixl.li

  • [hbrowse] remove module

  • [nitter] remove nitter.lacontrevoie.fr

1.26.6 - 2024-01-06

Extractors

Additions

Fixes

  • [gelbooru] fix favorite extractor (#4903)

  • [idolcomplex] fix extraction & update URL patterns (#5002)

  • [imagechest] fix loading more than 10 images in a gallery (#4469)

  • [jpgfish] update domain

  • [komikcast] fix manga extractor (#5027)

  • [komikcast] update domain (#5027)

  • [lynxchan] update bbw-chan domain (#4970)

  • [manganelo] fix extraction & recognize .to TLDs (#5005)

  • [paheal] restore extension metadata (#4976)

  • [rule34us] add fallback for video-cdn1 videos (#4985)

  • [weibo] fix AttributeError in user extractor (#5022)

Improvements

  • [gelbooru] show error for invalid API responses (#4903)

  • [rule34] recognize URLs with www subdomain (#4984)

  • [twitter] raise error for invalid strategy values (#4953)

Metadata

  • [fanbox] add metadata option (#4921)

  • [nijie] add count metadata (#146)

  • [pinterest] add count metadata (#4981)

Miscellaneous

  • fix and update zsh completion (#4972)

  • fix --cookies-from-browser macOS Firefox profile path

1.26.5 - 2023-12-23

Extractors

Additions

  • [deviantart] add intermediary option (#4955)

  • [inkbunny] add unread extractor (#4934)

  • [mastodon] support non-numeric status IDs (#4936)

  • [myhentaigallery] recognize /g/ URLs (#4920)

  • [postmill] add support (#4917, #4919)

  • {shimmie2[ support rule34hentai.net (#861, #4789, #4945)

Fixes

  • [deviantart] add workaround for integer client-id values (#4924)

  • [exhentai] fix error for infinite fallback-retries (#4911)

  • [inkbunny] stop pagination on empty results

  • [patreon] fix bootstrap data extraction again (#4904)

  • [tumblr] fix exception after waiting for rate limit (#4916)

Improvements

  • [exhentai] output continuation URL when interrupted (#4782)

  • [inkbunny] improve /submissionsviewall.php patterns (#4934)

  • [tumblr] support infinite fallback-retries

  • [twitter] default to tweets timeline when replies are enabled (#4953)

Metadata

  • [danbooru] provide tags as list (#4942)

  • [deviantart] set is_original for intermediary URLs to false

  • [twitter] remove date_liked (#3850, #4108, #4657)

Docker

  • add Docker instructions to README (#4850)

  • fix auto-generation of latest tags

1.26.4 - 2023-12-10

Extractors

Additions

Fixes

  • [nijie] fix image URLs of multi-image posts (#4876)

  • [patreon] fix bootstrap data extraction (#4904, #4906)

  • [twitter] fix /media timelines (#4898, #4899)

  • [twitter] retry API requests when response contains incomplete results (#4811)

Improvements

  • [exhentai] store more cookies when logging in with username & password (#4881)

  • [twitter] generalize "Login Required" errors (#4734, #4324)

Options

  • add -e/--error-file command-line and output.errorfile config option (#4732)

Miscellaneous

  • automatically build and push Docker images

  • prompt for passwords on login when necessary

  • fix util.dump_response() to work with bytes header values

1.26.3 - 2023-11-27

Extractors

Additions

  • [behance] support text modules (#4799)

  • [behance] add modules option (#4799)

  • [blogger] support www.micmicidol.club (#4759)

  • [erome] add count metadata (#4812)

  • [exhentai] add gp option (#4576)

  • [fapello] support .su TLD (#4840, #4841)

  • [pixeldrain] add file and album extractors (#4839)

  • [pixeldrain] add api-key option (#4839)

  • [tmohentai] add gallery extractor (#4808, #4832)

Fixes

  • [cyberdrop] update to site layout changes

  • [exhentai] handle Downloading … requires GP errors (#4576, #4763)

  • [exhentai] fix empty API URL with "source": "hitomi" (#4829)

  • [hentaifoundry] check for and update expired sessions (#4694)

  • [hiperdex] fix manga metadata

  • [idolcomplex] update to site layout changes

  • [imagefap] fix resolution of single images

  • [instagram] fix exception on empty video_versions (#4795)

  • [mangaread] fix extraction

  • [mastodon] fix reblogs (#4580)

  • [nitter] fix video extraction (#4853, #4855)

  • [pornhub] fix user metadata for gifs

  • [tumblr] fix day extractor

  • [wallpapercave] fix extraction

  • [warosu] fix file URLs

  • [webtoons] fix pagination when receiving an HTTP redirect

  • [xvideos] fix metadata extraction

  • [zerochan] fix metadata extraction

Improvements

  • [hentaicosplays] force https:// for download URLs

  • [oauth] warn when cache is enabled but not writeable (#4771)

  • [sankaku] update URL patterns

  • [twitter] ignore promoted Tweets (#3894, #4790)

  • [weibo] detect redirects to login page (#4773)

Removals

  • [foolslide] remove powermanga.org

Downloaders

Changes

  • [http] treat files not passing filesize-min/-max as skipped (#4821)

Options

Additions

  • add metadata-extractor option (#4549)

  • support metadata-* names for *-metadata options (for example url-metadata is now also recognized as metadata-url)

CLI

Additions

  • implement -I/--input-file-comment and -x/--input-file-delete options (#4732)

  • add --ugoira as a general version of --ugoira-conv and co.

  • add --mtime as a general version of --mtime-from-date

  • add --cbz

Fixes

  • allow --mtime-from-date to work with Weibo`s metadata structure

Miscellaneous

Additions

  • add a simple Dockerfile (#4831)

1.26.2 - 2023-11-04

Extractors

Additions

  • [4archive] add thread and board extractors (#1262, #2418, #4400, #4710, #4714)

  • [hitomi] recognize imageset gallery URLs (#4756)

  • [kemonoparty] add revision_index metadata field (#4727)

  • [misskey] support misskey.design (#4713)

  • [reddit] support Reddit Mobile share links (#4693)

  • [sankaku] support /posts/ tag search URLs (#4740)

  • [twitter] recognize fixupx.com URLs (#4755)

Fixes

  • [exhentai] update to site layout changes (#4730, #4754)

  • [exhentai] provide fallback URLs (#1021, #4745)

  • [exhentai] disable DH ciphers to avoid DH_KEY_TOO_SMALL errors (#1021, #4593)

  • [idolcomplex] disable sending Referer headers (#4726)

  • [instagram] update API headers

  • [kemonoparty] fix parsing of non-standard date values (#4676)

  • [patreon] fix campaign_id extraction (#4699, #4715, #4736, #4738)

  • [pixiv] load cookies for non-OAuth URLs (#4760)

  • [twitter] fix avatars without date information (#4696)

  • [twitter] restore truncated retweet texts (#3430, #4690)

  • [weibo] fix Sina Visitor requests

Improvements

  • [behance] unescape embed URLs (#4742)

  • [fantia] simplify tags to a list of strings (#4752)

  • [kemonoparty] limit title length (#4741)

  • [nijie] set 1-2s delay between requests to avoid 429 errors

  • [patreon] provide ways to manually specify a user's campaign_id

    • https://www.patreon.com/id:12345

    • https://www.patreon.com/USER?c=12345

    • https://www.patreon.com/USER?campaign_id=12345

  • [twitter] cache user_by_… results (#4719)

Post Processors

Fixes

  • [metadata] ignore non-string tag values (#4764)

Miscellaneous

Fixes

  • prevent crash when stdout.line_buffering is not defined (#642)

1.26.1 - 2023-10-21

Extractors

Additions

  • [bunkr] add extractor for media URLs (#4684)

  • [chevereto] add generic extractors for chevereto sites (#4664)

  • [deviantart] implement "group": "skip" (#4630)

  • [fantia] add content_count and content_num metadata fields (#4627)

  • [imgbb] add displayname and user_id metadata (#4626)

  • [kemonoparty] support post revisions; add revisions option (#4498, #4597)

  • [kemonoparty] support searches (#3385, #4057)

  • [kemonoparty] support discord URLs with channel IDs (#4662)

  • [moebooru] add metadata option (#4646)

  • [newgrounds] support multi-image posts (#4642)

  • [sankaku] support /posts/ URLs (#4688)

  • [twitter] add sensitive metadata field (#4619)

Fixes

  • [4chanarchives] disable Referer headers by default (#4686)

  • [bunkr] fix /d/ file URLs (#4685)

  • [deviantart] expand nested comment replies (#4653)

  • [deviantart] disable jwt (#4652)

  • [hentaifoundry] fix .swf file downloads (#4641)

  • [imgbb] fix user metadata extraction (#4626)

  • [imgbb] update pagination end condition (#4626)

  • [kemonoparty] update API endpoints (#4676, #4677)

  • [patreon] update campaign_id path (#4639)

  • [reddit] fix wrong previews (#4649)

  • [redgifs] fix niches extraction (#4666, #4667)

  • [twitter] fix crash due to missing source (#4620)

  • [warosu] fix extraction (#4634)

Post Processors

Additions

  • support {_filename}, {_directory}, and {_path} replacement fields for --exec (#4633)

Miscellaneous

Improvements

  • avoid temporary copies with --cookies-from-browser by opening cookie databases in read-only mode

1.26.0 - 2023-10-03

  • Extractors

    Additions

    • [behance] add date metadata field (#4417)

    • [danbooru] support booru.borvar.art (#4096)

    • [danbooru] support donmai.moe

    • [deviantart] add is_original metadata field (#4559)

    • [e621] support e6ai.net (#4320)

    • [exhentai] add fav option (#4409)

    • [gelbooru_v02] support xbooru.com (#4493)

    • [instagram] add following extractor (#1848)

    • [pillowfort] support /tagged/ URLs (#4570)

    • [pornhub] add gif support (#4463)

    • [reddit] add previews option (#4322)

    • [redgifs] add niches extractor (#4311, #4312)

    • [redgifs] support order parameter for user URLs (#4583)

    • [twitter] add user extractor and include option (#4275)

    • [twitter] add tweet-endpoint option (#4307)

    • [twitter] add date_original metadata for retweets (#4337, #4443)

    • [twitter] extract source metadata (#4459)

    • [twitter] support x.com URLs (#4452)

    Improvements

    • include Referer header in all HTTP requests (#4490, #4518) (can be disabled with referer option)

    • [behance] show errors for mature content (#4417)

    • [deviantart] re-add quality option and /intermediary/ transform

    • [fantia] improve metadata extraction (#4126)

    • [instagram] better error messages for invalid users (#4606)

    • [mangadex] support multiple values for lang (#4093)

    • [mastodon] support /@USER/following URLs (#4608)

    • [moebooru] match search URLs with empty tags (#4354)

    • [pillowfort] extract b2_lg_url media (#4570)

    • [reddit] improve comment metadata (#4482)

    • [reddit] ignore /message/compose URLs (#4482, #4581)

    • [redgifs] provide collection metadata as separate field (#4508)

    • [redgifs] match gfycat image URLs (#4558)

    • [twitter] improve error messages for single Tweets (#4369)

    Fixes

    • [acidimg] fix extraction

    • [architizer] fix extraction (#4537)

    • [behance] fix and update user extractor (#4417)

    • [behance] fix cookie usage (#4417)

    • [behance] handle videos without renditions (#4523)

    • [bunkr] fix media domain for cdn9 (#4386, #4412)

    • [bunkr] fix extracting .wmv files (#4419)

    • [bunkr] fix media domain for cdn-pizza.bunkr.ru (#4489)

    • [bunkr] fix extraction (#4514, #4532, #4529, #4540)

    • [deviantart] fix full resolution URLs for non-downloadable images (#293, #4548, #4563)

    • [deviantart] fix shortened URLs (#4316)

    • [deviantart] fix search (#4384)

    • [deviantart] update Eclipse API endpoints (#4553, #4615)

    • [deviantart] use private tokens for is_mature posts (#4563)

    • [flickr] update default API credentials (#4332)

    • [giantessbooru] fix extraction (#4373)

    • [hiperdex] fix crash for titles containing Unicode characters (#4325)

    • [hiperdex] fix manga metadata

    • [imagefap] fix pagination (#3013)

    • [imagevenue] fix extraction (#4473)

    • [instagram] fix private posts with long shortcodes (#4362)

    • [instagram] fix video preview archive IDs (#2135, #4455)

    • [instagram] handle exceptions due to missing media (#4555)

    • [issuu] fix extraction (#4420)

    • [jpgfish] update domain to jpg1.su (#4494)

    • [kemonoparty] update favorite API endpoint (#4522)

    • [lensdump] fix extraction (#4352)

    • [mangakakalot] update domain

    • [reddit] fix preview.redd.it URLs (#4470)

    • [patreon] fix extraction (#4547)

    • [pixiv] handle errors for private novels (#4481)

    • [pornhub] fix extraction (#4301)

    • [pururin] fix extraction (#4375)

    • [subscribestar] fix preview detection (#4468)

    • [twitter] fix crash on private user (#4349)

    • [twitter] fix TweetWithVisibilityResults (#4369)

    • [twitter] fix crash when sortIndex is undefined (#4499)

    • [zerochan] fix tags extraction (#4315, #4319)

    Removals

    • [gfycat] remove module

    • [shimmie2] remove meme.museum

  • Post Processors

    Changes

    • update finalize events

      • add finalize-error and finalize-success events that trigger depending on whether error(s) did or did not happen

      • change finalize to always trigger regardless of error status

    Additions

    • add python post processor

    • add prepare-after event (#4083)

    • [ugoira] add "framerate": "uniform" (#4421)

    Improvements

    • [ugoira] extend ffmpeg-output (#4421)

    Fixes

    • [ugoira] restore libx264-prevent-odd (#4407)

    • [ugoira] fix high frame rates (#4421)

  • Downloaders

    Fixes

    • [http] close connection when file already exists (#4403)

  • Options

    Additions

    • support parent>child categories for child extractor options, for example an imgur album from a reddit thread with reddit>imgur

    • implement subconfigs option (#4440)

    • add "ascii+" as a special path-restrict value (#4371)

    Removals

    • remove pyopenssl option

  • Tests

    Improvements

    • move extractor results into their own, separate files (#4504)

    • include fallback URLs in content tests (#3163)

    • various test method improvements

  • Miscellaneous

    Fixes

    • [formatter] use value of last alternative (#4492)

    • fix imports when running __main__.py (#4581)

    • fix symlink resolution in __main__.py

    • fix default Firefox user agent string

1.25.8 - 2023-07-15

Changes

  • update default User-Agent header to Firefox 115 ESR

Additions

  • [gfycat] support @me user (#3770, #4271)

  • [gfycat] implement login support (#3770, #4271)

  • [reddit] notify users about registering an OAuth application (#4292)

  • [twitter] add ratelimit option (#4251)

  • [twitter] use TweetResultByRestId endpoint that allows accessing single Tweets without login (#4250)

Fixes

  • [bunkr] use .la TLD for media-files12 servers (#4147, #4276)

  • [erome] ignore duplicate album IDs

  • [fantia] send X-Requested-With header (#4273)

  • [gelbooru_v01] fix source metadata (#4302, #4303)

  • [gelbooru_v01] update vidyart domain

  • [jpgfish] update domain to jpeg.pet

  • [mangaread] fix tags metadata extraction

  • [naverwebtoon] fix comic metadata extraction

  • [newgrounds] extract & pass auth token during login (#4268)

  • [paheal] fix extraction (#4262, #4293)

  • [paheal] unescape source

  • [philomena] fix --range (#4288)

  • [philomena] handle 429 Too Many Requests errors (#4288)

  • [pornhub] set accessAgeDisclaimerPH cookie (#4301)

  • [reddit] use 0.6s delay between API requests (#4292)

  • [seiga] set skip_fetish_warning cookie (#4242)

  • [slideshare] fix extraction

  • [twitter] fix following extractor not getting all users (#4287)

  • [twitter] use GraphQL search endpoint by default (#4264)

  • [twitter] do not treat missing TimelineAddEntries instruction as fatal (#4278)

  • [weibo] fix cursor based pagination

  • [wikifeet] fix tag extraction (#4289, #4291)

Removals

  • [bcy] remove module

  • [lineblog] remove module

1.25.7 - 2023-07-02

Additions

  • [flickr] add 'exif' option

  • [flickr] add 'metadata' option (#4227)

  • [mangapark] add 'source' option (#3969)

  • [twitter] extend 'conversations' option (#4211)

Fixes

  • [furaffinity] improve 'description' HTML (#4224)

  • [gelbooru_v01] fix '--range' (#4167)

  • [hentaifox] fix titles containing '@' (#4201)

  • [mangapark] update to v5 (#3969)

  • [piczel] update API server address (#4244)

  • [poipiku] improve error detection (#4206)

  • [sankaku] improve warnings for unavailable posts

  • [senmanga] ensure download URLs have a scheme (#4235)

1.25.6 - 2023-06-17

Additions

  • [blogger] download files from lh*.googleusercontent.com (#4070)

  • [fantia] extract plan metadata (#2477)

  • [fantia] emit warning for non-visible content sections (#4128)

  • [furaffinity] extract favorite_id metadata (#4133)

  • [jschan] add generic extractors for jschan image boards (#3447)

  • [kemonoparty] support .su TLDs (#4139)

  • [pixiv:novel] add novel-bookmark extractor (#4111)

  • [pixiv:novel] add full-series option (#4111)

  • [postimage] add gallery support, update image extractor (#3115, #4134)

  • [redgifs] support galleries (#4021)

  • [twitter] extract conversation_id metadata (#3839)

  • [vipergirls] add login support (#4166)

  • [vipergirls] use API endpoints (#4166)

  • [formatter] implement H conversion (#4164)

Fixes

  • [acidimg] fix extraction (#4136)

  • [bunkr] update domain to bunkrr.su (#4159, #4189)

  • [bunkr] fix video downloads

  • [fanbox] prevent exception due to missing embeds (#4088)

  • [instagram] fix retrieving /tagged posts (#4122)

  • [jpgfish] update domain to jpg.pet (#4138)

  • [pixiv:novel] fix error with embeds extraction (#4175)

  • [pornhub] improve redirect handling (#4188)

  • [reddit] fix crash due to empty crosspost_parent_lists (#4120, #4172)

  • [redgifs] update search URL pattern (#4115, #4185)

  • [senmanga] fix and update (#4160)

  • [twitter] use GraphQL API search endpoint (#3942)

  • [wallhaven] improve HTTP error handling (#4192)

  • [weibo] prevent fatal exception due to missing video data (#4150)

  • [weibo] fix .json extension for some videos

1.25.5 - 2023-05-27

Additions

  • [8muses] add parts metadata field (#3329)

  • [danbooru] add date metadata field (#4047)

  • [e621] add date metadata field (#4047)

  • [gofile] add basic password support (#4056)

  • [imagechest] implement API support (#4065)

  • [instagram] add order-files option (#3993, #4017)

  • [instagram] add order-posts option (#3993, #4017)

  • [instagram] add metadata option (#3107)

  • [jpgfish] add jpg.fishing extractors (#2657, #2719)

  • [lensdump] add lensdump.com extractors (#2078, #4104)

  • [mangaread] add mangaread.org extractors (#2425, #2781)

  • [misskey] add favorite extractor (#3950)

  • [pixiv] add novel support (#1241, #4044)

  • [reddit] support cross-posted media (#887, #3586, #3976)

  • [postprocessor:exec] support tilde expansion for command

  • [formatter] support slicing strings as bytes (#4087)

Fixes

  • [8muses] fix value of album[url] (#3329)

  • [danbooru] refactor pagination logic (#4002)

  • [fanbox] skip invalid posts (#4088)

  • [gofile] automatically fetch website-token

  • [kemonoparty] fix kemono and coomer logins sharing the same cache (#4098)

  • [newgrounds] add default delay between requests (#4046)

  • [nsfwalbum] detect placeholder images

  • [poipiku] extract full descriptions (#4066)

  • [tcbscans] update domain to tcbscans.com (#4080)

  • [twitter] extract TwitPic URLs in text (#3792, #3796)

  • [weibo] require numeric IDs to have length >= 10 (#4059)

  • [ytdl] fix crash due to removed no_color attribute

  • [cookies] improve logging behavior (#4050)

1.25.4 - 2023-05-07

Additions

  • [4chanarchives] add thread and board extractors (#4012)

  • [foolfuuka] add archive.palanq.win

  • [imgur] add favorite-folder extractor (#4016)

  • [mangadex] add status and tags metadata (#4031)

  • allow selecting a domain with --cookies-from-browser

  • add --cookies-export command-line option

  • add -C as short option for --cookies

  • include exception type in config error messages

Fixes

  • [exhentai] update sadpanda check

  • [imagechest] load all images when a "Load More" button is present (#4028)

  • [imgur] fix bug causing some images/albums from user profiles and favorites to be ignored

  • [pinterest] update endpoint for related board pins

  • [pinterest] fix pin.it extractor

  • [ytdl] fix yt-dlp --xff/--geo-bypass tests (#3989)

Removals

  • [420chan] remove module

  • [foolfuuka] remove archive.alice.al and tokyochronos.net

  • [foolslide] remove sensescans.com

  • [nana] remove module

1.25.3 - 2023-04-30

Additions

  • [imagefap] extract description and categories metadata (#3905)

  • [imxto] add gallery extractor (#1289)

  • [itchio] add game extractor (#3923)

  • [nitter] extract user IDs from encoded banner URLs

  • [pixiv] allow sorting search results by popularity (#3970)

  • [reddit] match preview.redd.it URLs (#3935)

  • [sankaku] support post URLs with MD5 hashes (#3952)

  • [shimmie2] add generic extractors for Shimmie2 sites (#3734, #943)

  • [tumblr] add day extractor (#3951)

  • [twitter] support profile-conversation entries (#3938)

  • [vipergirls] add thread and post extractors (#3812, #2720, #731)

  • [downloader:http] add consume-content option (#3748)

Fixes

  • [2chen] update domain to sturdychan.help

  • [behance] fix extraction (#3980)

  • [deviantart] retry downloads with private token (#3941)

  • [imagefap] fix empty tags metadata

  • [manganelo] support arbitrary minor version separators (#3972)

  • [nozomi] fix file URLs (#3925)

  • [oauth] catch exceptions from webbrowser.get() (#3947)

  • [pixiv] fix pixivision extraction

  • [reddit] ignore id-max value "zik0zj"/2147483647 (#3939, #3862, #3697, #3606, #3546, #3521, #3412)

  • [sankaku] sanitize date: tags (#1790)

  • [tumblr] fix and update pagination logic (#2191)

  • [twitter] fix user metadata when downloading quoted Tweets (#3922)

  • [ytdl] fix crash due to --geo-bypass deprecation (#3975)

  • [postprocessor:metadata] support putting keys in quotes

  • include more optional dependencies in executables (#3907)

1.25.2 - 2023-04-15

Additions

  • [deviantart] add public option

  • [nitter] extract videos from source elements (#3912)

  • [twitter] add date_liked and date_bookmarked metadata for liked and bookmarked Tweets (#3816)

  • [urlshortener] add support for bit.ly & t.co (#3841)

  • [downloader:http] add MIME type and signature for .heic files (#3915)

Fixes

  • [blogger] update regex to get the highest resolution URLs (#3863, #3870)

  • [bunkr] update domain to bunkr.la (#3813, #3877)

  • [deviantart] keep using private access tokens when requesting download URLs (#3845, #3857, #3896)

  • [hentaifoundry] fix content filters (#3887)

  • [hotleak] fix downloading of creators whose name starts with a category name (#3871)

  • [imagechest] fix extraction (#3914)

  • [realbooru] fix extraction (#2530)

  • [sexcom] fix pagination (#3906)

  • [sexcom] fix HD video extraction

  • [shopify] fix collection extractor (#3866, #3868)

  • [twitter] update to bookmark timeline v2 (#3859, #3854)

  • [twitter] warn about "withheld" Tweets and users (#3864)

Improvements

  • [danbooru] reduce number of API requests when fetching extended metadata

  • [deviantart:search] detect login redirects (#3860)

  • [generic] write regular expressions without x flags

  • [mastodon] try to get account IDs without access token

  • [twitter] calculate date from Tweet IDs

1.25.1 - 2023-03-25

Additions

  • [nitter] support nitter.it (#3819)

  • [twitter] add hashtag extractor (#3783)

  • [twitter] support Tweet content with >280 characters

  • [formatter] support loading f-strings from template files (#3800)

  • [formatter] support filesystem paths for \fM modules (#3399)

  • [formatter] support putting keys in quotes (e.g. user['name']) (#2559)

  • [postprocessor:metadata] add skip option (#3786)

Fixes

  • [output] set errors=replace for output streams (#3765)

  • [gelbooru] extract favorites without needing cookies (#3704)

  • [gelbooru] fix and improve --range for pools

  • [hiperdex] fix extraction (#3768)

  • [naverwebtoon] fix extraction (#3729)

  • [nitter] fix extraction for instances without user banners

  • [twitter] update API query hashes and parameters

  • [weibo] support mix_media_info entries (#3793)

  • fix circular reference detection for -K

Changes

  • update globals instead of overwriting the default (#3773)

1.25.0 - 2023-03-11

Changes

  • [e621] split e621 extractors from danbooru module (#3425)

  • [deviantart] remove mature scraps warning (#3691)

  • [deviantart] use /collections/all endpoint for favorites (#3666, #3668)

  • [newgrounds] update default image and audio archive IDs to prevent ID overlap (#3681)

  • rename --ignore-config to --config-ignore

Extractors

  • [catbox] add file extractor (#3570)

  • [deviantart] add search extractor (#538, #1264, #2954, #2970, #3577)

  • [deviantart] add gallery-search extractor (#1695)

  • [deviantart] support fxdeviantart.com URLs (##3740)

  • [e621] implement notes and pools metadata extraction (#3425)

  • [gelbooru] add favorite extractor (#3704)

  • [imagetwist] support phun.imagetwist.com and imagehaha.com domains (#3622)

  • [instagram] add user metadata field (#3107)

  • [manganelo] update and fix metadata extraction

  • [manganelo] support mobile-only chapters

  • [mangasee] extract author and genre metadata (#3703)

  • [misskey] add misskey extractors (#3717)

  • [pornpics] add gallery and search extractors (#263, #3544, #3654)

  • [redgifs] support v3 URLs (#3588. #3589)

  • [redgifs] add collection extractors (#3427, #3662)

  • [shopify] support ohpolly.com (#440, #3596)

  • [szurubooru] add tag and post extractors (#3583, #3713)

  • [twitter] add transform option

Options

  • [postprocessor:metadata] add sort and separators options

  • [postprocessor:exec] implement archive options (#3584)

  • add --config-create command-line option (#2333)

  • add --config-toml command-line option to load config files in TOML format

  • add output.stdout, output.stdin, and output.stderr options (#1621, #2152, #2529)

  • add hash_md5 and hash_sha1 functions (#3679)

  • implement globals option to enable defining custom functions for eval statements

  • implement archive-pragma option to use SQLite PRAGMA statements

  • implement actions to trigger events on logging messages (#3338, #3630)

  • implement ability to load external extractor classes

    • -X/--extractors command-line options

    • extractor.modules-sources config option

Fixes

  • [bunkr] fix extraction (#3636, #3655)

  • [danbooru] send gallery-dl User-Agent (#3665)

  • [deviantart] fix crash when handling deleted deviations in status updates (#3656)

  • [fanbox] fix crash with missing images (#3673)

  • [imagefap] update gallery URLs (#3595)

  • [imagefap] fix infinite pagination loop (#3594)

  • [imagefap] fix metadata extraction

  • [oauth] use default name for browsers without name attribute

  • [pinterest] unescape search terms (#3621)

  • [pixiv] fix --write-tags for "tags": "original" (#3675)

  • [poipiku] warn about incorrect passwords (#3646)

  • [reddit] update videos option (#3712)

  • [soundgasm] rewrite (#3578)

  • [telegraph] fix extraction when images are not in <figure> elements (#3590)

  • [tumblr] raise more detailed errors for dashboard-only blogs (#3628)

  • [twitter] fix some original retweets not downloading (#3744)

  • [ytdl] fix --parse-metadata (#3663)

  • [downloader:ytdl] prevent exception on empty results

Improvements

  • [downloader:http] use time.monotonic()

  • [downloader:http] update _http_retry to accept a Python function (#3569)

  • [postprocessor:metadata] speed up JSON encoding

  • replace json.loads/dumps with direct calls to JSONDecoder.decode/JSONEncoder.encode

  • improve option.Formatter performance

Removals

  • [nitter] remove nitter.pussthecat.org

1.24.5 - 2023-01-28

Additions

  • [booru] add url option

  • [danbooru] extend metadata option (#3505)

  • [deviantart] add extractor for status updates (#3539, #3541)

  • [deviantart] add support for /deviation/ and fav.me URLs (#3558, #3560)

  • [kemonoparty] extract hash metadata for discord files (#3531)

  • [lexica] add search extractor (#3567)

  • [mastodon] add num and count metadata fields (#3517)

  • [nudecollect] add image and album extractors (#2430, #2818, #3575)

  • [wikifeet] add gallery extractor (#519, #3537)

  • [downloader:http] add signature checks for .blend, .obj, and .clip files (#3535)

  • add extractor.retry-codes option

  • add -O/--postprocessor-option command-line option (#3565)

  • improve write-pages output

Fixes

  • [bunkr] fix downloading .mkv and .ts files (#3571)

  • [fantia] send X-CSRF-Token headers (#3576)

  • [generic] fix regex for non-src image URLs (#3555)

  • [hiperdex] update domain (#3572)

  • [hotleak] fix video URLs (#3516, #3525, #3563, #3581)

  • [instagram] always show cursor value after errors (#3440)

  • [instagram] update API domain, headers, and csrf token handling

  • [oauth] show client-id/api-key values (#3518)

  • [philomena] match URLs with www subdomain

  • [sankaku] update URL pattern (#3523)

  • [twitter] refresh guest tokens (#3445, #3458)

  • [twitter] fix search pagination (#3536, #3534, #3549)

  • [twitter] use "browser": "firefox" by default (#3522)

1.24.4 - 2023-01-11

Additions

  • [downloader:http] add validate option

Fixes

  • [kemonoparty] fix regression from commit 473bd380 (#3519)

1.24.3 - 2023-01-10

Additions

  • [danbooru] extract uploader metadata (#3457)

  • [deviantart] initial implementation of username & password login for scraps (#1029)

  • [fanleaks] add post and model extractors (#3468, #3474)

  • [imagefap] add folder extractor (#3504)

  • [lynxchan] support bbw-chan.nl (#3456, #3463)

  • [pinterest] support All Pins boards (#2855, #3484)

  • [pinterest] add domain option (#3484)

  • [pixiv] implement metadata-bookmark option (#3417)

  • [tcbscans] add chapter and manga extractors (#3189)

  • [twitter] implement syndication=extended (#3483)

  • implement slice notation for range options (#918, #2865)

  • allow filter options to be a list of expressions

Fixes

  • [behance] use delay between requests (#2507)

  • [bunkr] fix URLs returned by API (#3481)

  • [fanbox] return imageMap files in order (#2718)

  • [imagefap] use delay between requests (#1140)

  • [imagefap] warn about redirects to /human-verification (#1140)

  • [kemonoparty] reject invalid/empty files (#3510)

  • [myhentaigallery] handle whitespace before title tag (#3503)

  • [poipiku] fix extraction for a different warning button style (#3493, #3460)

  • [poipiku] warn about login requirements

  • [telegraph] fix file URLs (#3506)

  • [twitter] fix crash when using expand and syndication (#3473)

  • [twitter] apply tweet type checks before uniqueness check (#3439, #3455)

  • [twitter] force https:// for TwitPic URLs (#3449)

  • [ytdl] adapt to yt-dlp changes

  • update and improve documentation (#3453, #3462, #3496)

1.24.2 - 2022-12-18

Additions

  • [2chen] support .club URLs (#3406)

  • [deviantart] extract sta.sh URLs from text_content (#3366)

  • [deviantart] add /view URL support (#3367)

  • [e621] implement threshold option to control pagination (#3413)

  • [fapello] add post, user and path extractors (#3065, #3360, #3415)

  • [imgur] add support for imgur.io URLs (#3419)

  • [lynxchan] add generic extractors for lynxchan imageboards (#3389, #3394)

  • [mangafox] extract more metadata (#3167)

  • [pixiv] extract date_url metadata (#3405)

  • [soundgasm] add audio and user extractors (#3384, #3388)

  • [webmshare] add video extractor (#2410)

  • support Firefox containers for --cookies-from-browser (#3346)

Fixes

  • [2chen] fix file URLs

  • [bunkr] update domain (#3391)

  • [exhentai] fix pagination

  • [imagetwist] fix extraction

  • [imgth] rewrite

  • [instagram] prevent post date overwriting file date (#3392)

  • [khinsider] fix metadata extraction

  • [komikcast] update domain and fix extraction

  • [reddit] increase id-max default value (#3397)

  • [seiga] raise error when redirected to login page (#3401)

  • [sexcom] fix video URLs (#3408, #3414)

  • [twitter] update search pagination (#544)

  • [warosu] fix and update

  • [zerochan] update for layout v3

  • restore paths for archived files (#3362, #3377)

  • use util.NONE as keyword-default default value (#3334)

Removals

  • [foolslide] remove kireicake

  • [kissgoddess] remove module

1.24.1 - 2022-12-04

Additions

  • [artstation] add pro-first option (#3273)

  • [artstation] add max-posts option (#3270)

  • [fapachi] add post and user extractors (#3339, #3347)

  • [inkbunny] provide additional metadata (#3274)

  • [nitter] add retweets option (#3278)

  • [nitter] add videos option (#3279)

  • [nitter] support /i/web/ and /i/user/ URLs (#3310)

  • [pixhost] add gallery support (#3336, #3353)

  • [weibo] add count metadata field (#3305)

  • [downloader:http] add retry-codes option (#3313)

  • [formatter] implement S format specifier to sort lists (#3266)

  • implement version-metadata option (#3201)

Fixes

  • [2chen] fix extraction (#3354, #3356)

  • [bcy] fix JSONDecodeError (#3321)

  • [bunkr] fix video downloads (#3326, #3335)

  • [bunkr] use media-files servers for more file types

  • [itaku] remove Extreme rating (#3285, #3287)

  • [hitomi] apply format check for every image (#3280)

  • [hotleak] fix UnboundLocalError (#3288, #3293)

  • [nitter] sanitize filenames (#3294)

  • [nitter] retry downloads on 404 (#3313)

  • [nitter] set hlsPlayback cookie

  • [patreon] fix 403 Forbidden errors (#3341)

  • [patreon] improve campaign_id extraction (#3235)

  • [patreon] update API query parameters

  • [pixiv] preserve tags order (#3266)

  • [reddit] use dash_url for videos (#3258, #3306)

  • [twitter] fix error when using user IDs for suspended accounts

  • [weibo] fix bug with empty playback_list (#3301)

  • [downloader:http] fix potential ZeroDivisionError (#3328)

Removals

  • [lolisafe] remove zz.ht

1.24.0 - 2022-11-20

Additions

  • [exhentai] add metadata to search results (#3181)

  • [gelbooru_v02] implement notes extraction

  • [instagram] add guide extractor (#3192)

  • [lolisafe] add support for xbunkr (#3153, #3156)

  • [mastodon] add instance_remote metadata field (#3119)

  • [nitter] add extractors for Nitter instances (#2415, #2696)

  • [pixiv] add support for new daily AI rankings category (#3214, #3221)

  • [twitter] add avatar and background extractors (#349, #3023)

  • [uploadir] add support for uploadir.com (#3162)

  • [wallhaven] add user extractor (#3212, #3213, #3226)

  • [downloader:http] add chunk-size option (#3143)

  • [downloader:http] add file signature check for .mp4 files

  • [downloader:http] add file signature check and MIME type for .avif files

  • [postprocessor] implement post-after event (#3117)

  • [postprocessor:metadata] implement "mode": "jsonl"

  • [postprocessor:metadata] add open, encoding, and private options

  • add --chunk-size command-line option (#3143)

  • add --user-agent command-line option

  • implement http-metadata option

  • implement "user-agent": "browser" (#2636)

Changes

  • [deviantart] restore cookies warning for mature scraps (#3129)

  • [instagram] use REST API for unauthenticated users by default

  • [downloader:http] increase default chunk-size to 32768 bytes (#3143)

  • build Windows executables using py2exe's new freeze() API

  • build executables on GitHub Actions with Python 3.11

  • reword error text for unsupported URLs

Fixes

  • [exhentai] fix pagination (#3181)

  • [khinsider] fix extraction (#3215, #3219)

  • [realbooru] fix download URLs (#2530)

  • [realbooru] fix tags extraction (#2530)

  • [tumblr] fall back to gifv when possible (#3095, #3159)

  • [twitter] fix login (#3220)

  • [twitter] update URL for syndication API (#3160)

  • [weibo] send Referer headers (#3188)

  • [ytdl] update parse_bytes location (#3256)

Improvements

  • [imxto] extract additional metadata (#3118, #3175)

  • [instagram] allow downloading avatars for private profiles (#3255)

  • [pixiv] raise error for invalid search/ranking parameters (#3214)

  • [twitter] update bookmarks pagination (#3172)

  • [downloader:http] refactor file signature checks

  • [downloader:http] improve -r/--limit-rate accuracy (#3143)

  • add loaded config files to debug output

  • improve -K output for lists

Removals

  • [instagram] remove login support (#3139, #3141, #3191)

  • [instagram] remove channel extractor

  • [ngomik] remove module

1.23.5 - 2022-10-30

Fixes

  • [instagram] fix AttributeError on user stories extraction (#3123)

1.23.4 - 2022-10-29

Additions

  • [aibooru] add support for aibooru.online (#3075)

  • [instagram] add 'avatar' extractor (#929, #1097, #2992)

  • [instagram] support 'instagram.com/s/' highlight URLs (#3076)

  • [instagram] extract 'coauthors' metadata (#3107)

  • [mangasee] add support for 'mangalife' (#3086)

  • [mastodon] add 'bookmark' extractor (#3109)

  • [mastodon] support cross-instance user references and '/web/' URLs (#3109)

  • [moebooru] implement 'notes' extraction (#3094)

  • [pixiv] extend 'metadata' option (#3057)

  • [reactor] match 'best', 'new', 'all' URLs (#3073)

  • [smugloli] add 'smugloli' extractors (#3060)

  • [tumblr] add 'fallback-delay' and 'fallback-retries' options (#2957)

  • [vichan] add generic extractors for vichan imageboards

Fixes

  • [bcy] fix extraction (#3103)

  • [gelbooru] support alternate parameter order in post URLs (#2821)

  • [hentai2read] support minor versions in chapter URLs (#3089)

  • [hentaihere] support minor versions in chapter URLs

  • [kemonoparty] fix 'dms' extraction (#3106)

  • [kemonoparty] update pagination offset

  • [manganelo] update domain to 'chapmanganato.com' (#3097)

  • [pixiv] use 'exact_match_for_tags' as default search mode (#3092)

  • [redgifs] fix 'token' extraction (#3080, #3081)

  • [skeb] fix extraction (#3112)

  • improve compatibility of DownloadArchive (#3078)

1.23.3 - 2022-10-15

Additions

  • [2chen] Add 2chen.moe extractor (#2707)

  • [8chan] add thread and board extractors (#2938)

  • [deviantart] add group option (#3018)

  • [fanbox] add content metadata field (#3020)

  • [instagram] restore cursor functionality (#2991)

  • [instagram] restore warnings for private profiles (#3004, #3045)

  • [nana] add nana extractors (#2967)

  • [nijie] add feed and followed extractors (#3048)

  • [tumblr] support https://www.tumblr.com/BLOGNAME URLs (#3034)

  • [tumblr] add offset option

  • [vk] add tagged extractor (#2997)

  • add path-extended option (#3021)

  • emit debug logging messages before calling time.sleep() (#2982)

Changes

  • [postprocessor:metadata] assume "mode": "custom" when format is given

Fixes

  • [artstation] skip missing projects (#3016)

  • [danbooru] fix ugoira metadata extraction (#3056)

  • [deviantart] fix deviation extraction (#2981)

  • [hitomi] fall back to webp when selected format is not available (#3030)

  • [imagefap] fix and improve folder extraction and gallery pagination (#3013)

  • [instagram] fix login (#3011, #3015)

  • [nozomi] fix extraction (#3051)

  • [redgifs] fix extraction (#3037)

  • [tumblr] sleep between fallback retries (#2957)

  • [vk] unescape error messages

  • fix duplicated metadata bug with -j (#3033)

  • fix bug when processing input file comments (#2808)

1.23.2 - 2022-10-01

Additions

  • [artstation] support search filters (#2970)

  • [blogger] add label and query metadata fields (#2930)

  • [exhentai] add a slash to the end of gallery URLs (#2947)

  • [instagram] add count metadata field (#2979)

  • [instagram] add api option

  • [kemonoparty] add count metadata field (#2952)

  • [mastodon] warn about moved accounts (#2939)

  • [newgrounds] add games extractor (#2955)

  • [newgrounds] extract type metadata

  • [pixiv] add series extractor (#2964)

  • [sankaku] implement refresh option (#2958)

  • [skeb] add search extractor and filters option (#2945)

Fixes

  • [deviantart] fix extraction (#2981, #2983)

  • [fappic] fix extraction

  • [instagram] extract higher-resolution photos (#2666)

  • [instagram] fix username and fullname metadata for saved posts (#2911)

  • [instagram] update API headers

  • [kemonoparty] send Referer headers (#2989, #2990)

  • [kemonoparty] restore favorites API endpoints (#2994)

  • [myportfolio] use fallback when no images are found (#2959)

  • [plurk] fix extraction (#2977)

  • [sankaku] detect expired links (#2958)

  • [tumblr] retry extraction of failed higher-resolution images (#2957)

1.23.1 - 2022-09-18

Additions

  • [flickr] add support for secure.flickr.com URLs (#2910)

  • [hotleak] add hotleak extractors (#2890, #2909)

  • [instagram] add highlight_title and date metadata for highlight downloads (#2879)

  • [paheal] add support for videos (#2892)

  • [tumblr] fetch high-quality inline images (#2877)

  • [tumblr] implement ratelimit option (#2919)

  • [twitter] add general support for unified cards (#2875)

  • [twitter] implement cards-blacklist option (#2875)

  • [zerochan] add metadata option (#2861)

  • [postprocessor:zip] implement files option (#2872)

Fixes

  • [bunkr] fix extraction (#2903)

  • [bunkr] use media-files servers for m4v and mov downloads (#2925)

  • [exhentai] improve 509.gif detection (#2901)

  • [exhentai] guess extension for original files (#2842)

  • [poipiku] use img-org.poipiku.com as image domain (#2796)

  • [reddit] prevent exception with empty submission URLs (#2913)

  • [redgifs] fix download URLs (#2884)

  • [smugmug] update default API credentials (#2881)

  • [twitter] provide proper date for syndication results (#2920)

  • [twitter] fix new-style /card_img/ URLs

  • remove all whitespace before comments after input file URLs (#2808)

1.23.0 - 2022-08-28

Changes

  • [twitter] update user and author metdata fields

    • for URLs with a single username or ID like https://twitter.com/USER or a search with a single from: statement, user will now always refer to the user referenced in the URL.

    • for all other URLs like https://twitter.com/i/bookmarks, user and author refer to the same user

    • author will always refer to the original Tweet author

  • [twitter] update quote_id and quote_by metadata fields

    • quote_id is now non-zero for quoted Tweets and contains the Tweet ID of the quotng Tweet (was the other way round before)

    • quote_by is only defined for quoted Tweets like before, but now contains the screen name of the user quoting this Tweet

  • [skeb] improve archive IDs for thumbnails and article images

Additions

  • [artstation] add num and count metadata fields (#2764)

  • [catbox] add album extractor (#2410)

  • [blogger] emit metadata for posts without files (#2789)

  • [foolfuuka] update supported domains

  • [gelbooru] add support for api_key and user_id (#2767)

  • [gelbooru] implement pagination for pool results (#2853)

  • [instagram] add support for a user's saved collections (#2769)

  • [instagram] provide date for directory format strings (#2830)

  • [kemonoparty] add favorites option (#2826, #2831)

  • [oauth] add host config option (#2806)

  • [rule34] implement pagination for pool results (#2853)

  • [skeb] add option to download article images (#1031)

  • [tumblr] download higher-quality images (#2761)

  • [tumblr] add count metadata field (#2804)

  • [wallhaven] implement metadata option (#2803)

  • [zerochan] add tag and image extractors (#1434)

  • [zerochan] implement login with username & password (#1434)

  • [postprocessor:metadata] implement mode: modify and mode: delete (#2640)

  • [formatter] add g conversion for slugifying a string (#2410)

  • [formatter] apply :J only to lists (#2833)

  • implement path-metadata option (#2734)

  • allow comments after input file URLs (#2808)

  • add global warnings option to control urllib3 warning behavior (#2762)

Fixes

  • [bunkr] fix extraction (#2788)

  • [deviantart] use public access token for journals (#2702)

  • [e621] fix extraction of popular posts

  • [fanbox] download cover images in original size (#2784)

  • [mastodon] allow downloading without access token (#2782)

  • [hitomi] update cache expiry time (#2863)

  • [hitomi] fix error when number of tag results is a multiple of 25 (#2870)

  • [mangahere] fix page-reverse option (#2795)

  • [poipiku] fix posts with more than one image (#2796)

  • [poipiku] update filter for static images (#2796)

  • [slideshare] fix metadata extraction

  • [twitter] unescape + in search queries (#2226)

  • [twitter] fall back to unfiltered search (#2766)

  • [twitter] ignore invalid user entries (#2850)

  • [vk] prevent exceptions for broken/invalid photos (#2774)

  • [vsco] fix collection extraction

  • [weibo] prevent exception for missing playback_list (#2792)

  • [weibo] prevent errors when paginating over album entries (#2817)

1.22.4 - 2022-07-15

Additions

  • [instagram] add pinned metadata field (#2752)

  • [itaku] categorize sections by group (#1842)

  • [khinsider] extract platform metadata

  • [tumblr] support /blog/view URLs (#2760)

  • [twitter] implement strategy option (#2712)

  • [twitter] add count metadata field (#2741)

  • [formatter] implement O format specifier (#2736)

  • [postprocessor:mtime] add value option (#2739)

  • add --no-postprocessors command-line option (#2725)

  • implement format-separator option (#2737)

Changes

  • [pinterest] handle section pins with separate extractors (#2684)

  • [postprocessor:ugoira] enable mtime by default (#2714)

Fixes

  • [bunkr] fix extraction (#2732)

  • [hentaifoundry] fix metadata extraction

  • [itaku] fix user caching (#1842)

  • [itaku] fix date parsing

  • [kemonoparty] ensure all files have an extension (#2740)

  • [komikcast] update domain

  • [mangakakalot] update domain

  • [newgrounds] only attempt to login if necessary (#2715)

  • [newgrounds] prevent exception on empty results (#2727)

  • [nozomi] reduce memory consumption during searches (#2754)

  • [pixiv] fix default background filenames

  • [sankaku] rewrite file URLs to s.sankakucomplex.com (#2746)

  • [slideshare] fix description extraction

  • [twitter] ignore previously seen Tweets (#2712)

  • [twitter] unescape HTML entities in content (#2757)

  • [weibo] handle invalid or broken status objects

  • [postprocessor:zip] ensure target directory exists (#2758)

  • make brotli an optional dependency (#2716)

  • limit path length for --write-pages output on Windows (#2733)

Removals

  • [foolfuuka] remove archive.wakarimasen.moe

1.22.3 - 2022-06-28

Changes

  • [twitter] revert strategy changes for user URLs (#2712, #2710)

  • update default User-Agent headers

1.22.2 - 2022-06-27

Additions

  • [cyberdrop] add fallback URLs (#2668)

  • [horne] add support for horne.red (#2700)

  • [itaku] add gallery and image extractors (#1842)

  • [poipiku] add user and post extractors (#1602)

  • [skeb] add following extractor (#2698)

  • [twitter] implement expand option (#2665)

  • [twitter] implement csrf option (#2676)

  • [unsplash] add collection_title and collection_id metadata fields (#2670)

  • [weibo] support tabtype=video listings (#2601)

  • [formatter] implement slice operator as format specifier

  • support cygwin/BSD/etc for --cookies-from-browser

Fixes

  • [instagram] improve metadata generated by _parse_post_api() (#2695, #2660)

  • [instagram} fix tag extractor (#2659)

  • [instagram] automatically invalidate expired login sessions

  • [twitter] fix pagination for conversion tweets

  • [twitter] improve "replies": "self" (#2665)

  • [twitter] improve strategy for user URLs (#2665)

  • [vk] take URLs from *_src entries (#2535)

  • [weibo] fix URLs generated by user extractor (#2601)

  • [weibo] fix retweets (#2601)

  • [downloader:ytdl] update _set_outtmpl() (#2692)

  • [formatter] fix !j conversion for non-serializable types (#2624)

  • [snap] Fix missing libslang dependency (#2655)

1.22.1 - 2022-06-04

Additions

  • [gfycat] add support for collections (#2629)

  • [instagram] support specifying users by ID

  • [paheal] extract more metadata (#2641)

  • [reddit] add home extractor (#2614)

  • [weibo] support usernames in URLs (#1662)

  • [weibo] support livephoto and gif files (#2146)

  • [weibo] add support for several different tabtype listings (#686, #2601)

  • [postprocessor:metadata] write to stdout by setting filename to "-" (#2624)

  • implement output.ansi option (#2628)

  • support user-defined output.mode settings (#2529)

Changes

  • [readcomiconline] remove default browser setting (#2625)

  • [weibo] switch to desktop API (#2601)

  • fix command-line argument name of --cookies-from-browser (#1606, #2630)

Fixes

  • [bunkr] change domain to app.bunkr.is (#2634)

  • [deviantart] fix folder listings with "pagination": "manual" (#2488)

  • [gofile] fix 401 Unauthorized errors (#2632)

  • [hypnohub] move to gelbooru_v02 instances (#2631)

  • [instagram] fix and update extractors (#2644)

  • [nozomi] remove slashes from search terms (#2653)

  • [pixiv] include .gif in background fallback URLs (#2495)

  • [sankaku] extend URL patterns (#2647)

  • [subscribestar] fix date metadata (#2642)

1.22.0 - 2022-05-25

Additions

  • [gelbooru_v01] add favorite extractor (#2546)

  • [Instagram] add tagged_users to keywords for stories (#2582, #2584)

  • [lolisafe] implement domain option (#2575)

  • [naverwebtoon] support (best)challenge comics (#2542)

  • [nijie] support /history_nuita.php listings (#2541)

  • [pixiv] provide more data when metadata is enabled (#2594)

  • [shopify] support several more sites by default (#2089)

  • [twitter] extract alt texts as description (#2617)

  • [twitter] recognize vxtwitter URLs (#2621)

  • [weasyl] implement metadata option (#2610)

  • implement --cookies-from-browser (#1606)

  • implement output.colors options (#2532)

  • implement string literals in replacement fields

  • support using extended format strings for archive keys

Changes

  • [foolfuuka] match 4chan filenames (#2577)

  • [pixiv] implement include option

    • provide avatar/background downloads as separate extractors (#2495)

  • [twitter] use a better strategy for user URLs

  • [twitter] disable cards by default

  • delay directory creation (#2461, #2474)

  • flush writes to stdout/stderr (#2529)

  • build executables on GitHub Actions with Python 3.10

Fixes

  • [artstation] use "browser": "firefox" by default (#2527)

  • [imgur] prevent exception with empty albums (#2557)

  • [instagram] report redirects to captcha challenges (#2543)

  • [khinsider] fix metadata extraction (#2611)

  • [mangafox] send Referer headers (#2592)

  • [mangahere] send Referer headers (#2592)

  • [mangasee] use randomly generated PHPSESSID cookie (#2560)

  • [pixiv] make retrieving ugoira metadata non-fatal (#2562)

  • [readcomiconline] update deobfuscation code (#2481)

  • [realbooru] fix extraction (#2530)

  • [vk] handle photos without width/height info (#2535)

  • [vk] fix user ID extraction (#2535)

  • [webtoons] extract real episode numbers (#2591)

  • create missing directories for archive files (#2597)

  • detect circular references with -K (#2609)

  • replace "\f" in --filename arguments with a form feed character (#2396)

Removals

1.21.2 - 2022-04-27

Additions

  • [deviantart] implement pagination option (#2488)

  • [pixiv] implement background option (#623, #1124, #2495)

  • [postprocessor:ugoira] report ffmpeg/mkvmerge errors (#2487)

Fixes

  • [cyberdrop] match cyberdrop.to URLs (#2496)

  • [e621] fix 403 errors (#2533)

  • [issuu] fix extraction (#2483)

  • [mangadex] download from available chapters despite externalUrl (#2503)

  • [photovogue] update domain and api endpoint (#2494)

  • [sexcom] add fallback for empty files (#2485)

  • [twitter] improve syndication video selection (#2354)

  • [twitter] fix various syndication issues (#2499, #2354)

  • [vk] fix extraction (#2512)

  • [weibo] fix infinite retries for deleted accounts (#2521)

  • [postprocessor:ugoira] use compatible paths with mkvmerge (#2487)

  • [postprocessor:ugoira] do not auto-select the image2 demuxer (#2492)

1.21.1 - 2022-04-08

Additions

  • [gofile] add gofile.io extractor (#2364)

  • [instagram] add previews option (#2135)

  • [kemonoparty] add duplicates option (#2440)

  • [pinterest] add extractor for created pins (#2452)

  • [pinterest] support multiple files per pin (#1619, #2452)

  • [telegraph] Add telegra.ph extractor (#2312)

  • [twitter] add syndication option (#2354)

  • [twitter] accept fxtwitter.com URLs (#2484)

  • [downloader:http] support using an arbitrary method and sending POST data (#2433)

  • [postprocessor:metadata] implement archive options (#2421)

  • [postprocessor:ugoira] add mtime option (#2307)

  • [postprocessor:ugoira] support setting timecodes with mkvmerge (#1550)

  • [formatter] support evaluating f-string literals

  • add --ugoira-conv-copy command-line option (#1550)

  • implement a contains() function for filter statements (#2446)

Fixes

  • [aryion] provide correct date metadata independent of DST

  • [furaffinity] fix search result pagination (#2402)

  • [hitomi] update and fix metadata extraction (#2444)

  • [kissgoddess] extract all images (#2473)

  • [mangasee] unescape manga names (#2454)

  • [newgrounds] update and fix pagination (#2456)

  • [newgrounds] warn about age-restricted posts (#2456)

  • [pinterest] do not force m3u8_native for video downloads (#2436)

  • [twibooru] fix posts without name (#2434)

  • [unsplash] replace dash with space in search API queries (#2429)

  • [postprocessor:mtime] fix timestamps from datetime objects (#2307)

  • fix yet another bug in _check_cookies() (#2372)

  • fix loading/storing cookies without domain

1.21.0 - 2022-03-14

Additions

  • [fantia] add num enumeration index (#2377)

  • [fantia] support "Blog Post" content (#2381)

  • [imagebam] add support for /view/ paths (#2378)

  • [kemonoparty] match beta.kemono.party URLs (#2348)

  • [kissgoddess] add gallery and model extractors (#1052, #2304)

  • [mememuseum] add tag and post extractors (#2264)

  • [newgrounds] add post_url metadata field (#2328)

  • [patreon] add image_large file type (#2257)

  • [toyhouse] support art listings (#1546, #2331)

  • [twibooru] add extractors for searches, galleries, and posts (#2219)

  • [postprocessor:metadata] implement mtime option (#2307)

  • [postprocessor:mtime] add event option (#2307)

  • add fish shell completion (#2363)

  • add timedelta class to global namespace in filter expressions

Changes

  • [seiga] require authentication with user_session cookie (#2372)

    • remove username & password login due to 2FA

  • refactor proxy support (#2357)

    • allow gallery-dl proxy settings to overwrite environment proxies

    • allow specifying different proxies for data extraction and download

Fixes

  • [bunkr] fix mp4 downloads (#2239)

  • [fanbox] fetch data for each individual post (#2388)

  • [hentaicosplays] send Referer header (#2317)

  • [imagebam] set nsfw_inter cookie (#2334)

  • [kemonoparty] limit default filename length (#2373)

  • [mangadex] fix chapters without translatedLanguage (#2352)

  • [newgrounds] fix video descriptions (#2328)

  • [skeb] add sent-requests option (#2322, #2330)

  • [slideshare] fix extraction

  • [subscribestar] unescape attachment URLs (#2370)

  • [twitter] fix handling of 429 Too Many Requests responses (#2339)

  • [twitter] warn about age-restricted Tweets (#2354)

  • [twitter] handle Tweets with "softIntervention" entries

  • [twitter] update query hashes

  • fix another bug in _check_cookies() (#2160)

1.20.5 - 2022-02-14

Additions

  • [furaffinity] add layout option (#2277)

  • [lightroom] add Lightroom gallery extractor (#2263)

  • [reddit] support standalone submissions on personal user pages (#2301)

  • [redgifs] support i.redgifs.com URLs (#2300)

  • [wallpapercave] add extractor for images and search results (#2205)

  • add signals-ignore option (#2296)

Changes

  • [danbooru] merge danbooru and e621 extractors

    • support atfbooru (#2283)

    • remove support for old e621 tag search URLs

Fixes

  • [furaffinity] improve new/old layout detection (#2277)

  • [imgbox] fix ImgboxExtractor (#2281)

  • [inkbunny] rename search parameters to their API equivalents

  • [kemonoparty] handle files without names (#2276)

  • [twitter] fix extraction (#2275, #2295)

  • [vk] fix infinite pagination loops (#2297)

  • [downloader:ytdl] make ImportErrors non-fatal (#2273)

1.20.4 - 2022-02-06

Additions

  • [e621] add favorite extractor (#2250)

  • [hitomi] add format option (#2260)

  • [kohlchan] add Kohlchan extractors (#2251)

  • [sexcom] add pins extractor (#2265)

  • [twitter] add warnings option (#2258)

  • add ability to disable TLS 1.2 (#2243)

  • add examples for custom gelbooru instances (#2262)

Fixes

  • [bunkr] fix mp4 downloads (#2239)

  • [gelbooru] improve and fix pagination (#2230, #2232)

  • [hitomi] "fix" 403 errors (#2260)

  • [kemonoparty] fix downloading smaller text files (#2267)

  • [patreon] disable TLS 1.2 by default (#2249)

  • [twitter] restore errors for protected timelines etc (#2237)

  • [twitter] restore logout functionality (#1719)

  • [twitter] provide fallback URLs for card images

  • [weibo] update pagination code (#2244)

1.20.3 - 2022-01-26

Fixes

  • [kemonoparty] fix DMs extraction (#2008)

  • [twitter] fix crash on Tweets with deleted quotes (#2225)

  • [twitter] fix crash on suspended Tweets without legacy entry (#2216)

  • [twitter] fix crash on unified cards without type

  • [twitter] prevent crash on invalid/deleted Retweets (#2225)

  • [twitter] update query hashes

1.20.2 - 2022-01-24

Additions

  • [twitter] add event extractor (closes #2109)

  • [twitter] support image_carousel_website unified cards

  • add --source-address command-line option (#2206)

  • add environment variable syntax to formatting.md (#2065)

Changes

  • [twitter] changes to cards option

    • enable cards by default

    • require cards to be set to "ytdl" to invoke youtube-dl/yt-dlp on unsupported cards

Fixes

  • [blogger] support new image domain (#2204)

  • [gelbooru] improve video file detection (#2188)

  • [hitomi] fix tag extraction (#2189)

  • [instagram] fix highlights extraction (#2197)

  • [mangadex] re-enable warning for external chapters (#2193)

  • [newgrounds] set suitabilities filter before starting a search (#2173)

  • [philomena] fix search parameter escaping (#2215)

  • [reddit] allow downloading from quarantined subreddits (#2180)

  • [sexcom] extend URL pattern (#2220)

  • [twitter] update to GraphQL API (#2212)

1.20.1 - 2022-01-08

Additions

  • [newgrounds] add search extractor (#2161)

Changes

  • restore -d/--dest functionality from before 1.20.0 (#2148)

  • change short option for --directory to -D

Fixes

  • [gelbooru] handle changed API response format (#2157)

  • [hitomi] fix image URLs (#2153)

  • [mangadex] fix extraction (#2177)

  • [rule34] use https://api.rule34.xxx for API requests

  • fix cookie checks for patreon, fanbox, fantia

  • improve UNC path handling (#2126)

1.20.0 - 2021-12-29

Additions

  • [500px] add favorite extractor (#1927)

  • [exhentai] add source option

  • [fanbox] support pixiv redirects (#2122)

  • [inkbunny] add search extractor (#2094)

  • [kemonoparty] support coomer.party (#2100)

  • [lolisafe] add generic album extractor for lolisafe/chibisafe instances (#2038, #2105)

  • [rule34us] add tag and post extractors (#1527)

  • add a generic extractor (#735, #683)

  • add -d/--directory and -f/--filename command-line options

  • add --sleep-request and --sleep-extractor command-line options

  • allow specifying sleep-* options as string

Changes

  • [cyberdrop] include file ID in default filenames

  • [hitomi] disable metadata by default

  • [kemonoparty] use service as subcategory (#2147)

  • [kemonoparty] change default files order to attachments,file,inline (#1991)

  • [output] write download progress indicator to stderr

  • [ytdl] prefer yt-dlp over youtube-dl (#1850, #2028)

  • rename --write-infojson to --write-info-json

Fixes

  • [500px] create directories per photo

  • [artstation] create directories per asset (#2136)

  • [deviantart] use /browse/newest for most-recent searches (#2096)

  • [hitomi] fix image URLs

  • [instagram] fix error when PostPage data is not in GraphQL format (#2037)

  • [instagran] match post URLs with usernames (#2085)

  • [instagram] allow downloading specific stories (#2088)

  • [furaffinity] warn when no session cookies were found

  • [pixiv] respect date ranges in search URLs (#2133)

  • [sexcom] fix and improve embed extraction (#2145)

  • [tumblrgallery] fix extraction (#2112)

  • [tumblrgallery] improve id extraction (#2115)

  • [tumblrgallery] improve search pagination (#2132)

  • [twitter] include 4096x4096 as a default image fallback (#1881, #2107)

  • [ytdl] update argument parsing to latest yt-dlp changes (#2124)

  • handle UNC paths (#2113)

1.19.3 - 2021-11-27

Additions

  • [dynastyscans] add manga extractor (#2035)

  • [instagram] include user metadata for tagged downloads (#2024)

  • [kemonoparty] implement files option (#1991)

  • [kemonoparty] add dms option (#2008)

  • [mangadex] always provide artist, author, and group metadata fields (#2049)

  • [philomena] support furbooru.org (#1995)

  • [reactor] support thatpervert.com (#2029)

  • [shopify] support loungeunderwear.com (#2053)

  • [skeb] add thumbnails option (#2047, #2051)

  • [subscribestar] add num enumeration index (#2040)

  • [subscribestar] emit metadata for posts without media (#1569)

  • [ytdl] implement cmdline-args and config-file options to allow parsing ytdl command-line options (#1680)

  • [formatter] implement D format specifier

  • extend blacklist/whitelist syntax (#2025)

Fixes

  • [dynastyscans] provide date as datetime object (#2050)

  • [exhentai] fix extraction for disowned galleries (#2055)

  • [gelbooru] apply workaround for pagination limits

  • [kemonoparty] skip duplicate files (#2032, #1991, #1899)

  • [kemonoparty] provide date metadata for gumroad (#2007)

  • [mangoxo] fix metadata extraction

  • [twitter] distinguish between fatal & nonfatal errors (#2020)

  • [twitter] fix extractor for direct image links (#2030)

  • [webtoons] use download URLs that do not require a Referer header (#2005)

  • [ytdl] improve error handling (#1680)

  • [downloader:ytdl] prevent crash in _progress_hook() (#1680)

Removals

  • [seisoparty] remove module

1.19.2 - 2021-11-05

Additions

  • [kemonoparty] add comments option (#1980)

  • [skeb] add user and post extractors (#1031, #1971)

  • [twitter] add pinned option

  • support accessing environment variables and the current local datetime in format strings (#1968)

  • add special type format strings to docs (#1987)

Fixes

  • [cyberdrop] fix video extraction (#1993)

  • [deviantart] fix index values for stashed deviations

  • [gfycat] provide consistent userName values for user downloads (#1962)

  • [gfycat] show warning when there are no available formats

  • [hitomi] fix image URLs (#1975, #1982, #1988)

  • [instagram] update query hashes

  • [mangakakalot] update domain and fix extraction

  • [mangoxo] fix login and extraction

  • [reddit] prevent crash for galleries with no media_metadata (#2001)

  • [redgifs] update to API v2 (#1984)

  • fix calculating retry sleep times (#1990)

1.19.1 - 2021-10-24

Additions

  • [inkbunny] add following extractor (#515)

  • [inkbunny] add pool extractor (#1937)

  • [kemonoparty] add discord extractor (#1827, #1940)

  • [nhentai] add tag extractor (#1950, #1955)

  • [patreon] add files option (#1935)

  • [picarto] add gallery extractor (#1931)

  • [pixiv] add sketch extractor (#1497)

  • [seisoparty] add favorite extractor (#1906)

  • [twitter] add size option (#1881)

  • [vk] add album extractor (#474, #1952)

  • [postprocessor:compare] add equal option (#1592)

Fixes

  • [cyberdrop] extract direct download URLs (#1943)

  • [deviantart] update search argument handling (#1911)

  • [deviantart] full resolution for non-downloadable images (#293)

  • [furaffinity] unquote search queries (#1958)

  • [inkbunny] match "long" URLs for pools and favorites (#1937)

  • [kemonoparty] improve inline extraction (#1899)

  • [mangadex] update parameter handling for API requests (#1908)

  • [patreon] better filenames for content images (#1954)

  • [redgifs][gfycat] provide fallback URLs (#1962)

  • [downloader:ytdl] prevent crash in _progress_hook()

  • restore SOCKS support for Windows executables

1.19.0 - 2021-10-01

Additions

  • [aryion] add tag extractor (#1849)

  • [desktopography] implement desktopography extractors (#1740)

  • [deviantart] implement auto-unwatch option (#1466, #1757)

  • [fantia] add date metadata field (#1853)

  • [fappic] add image extractor (#1898)

  • [gelbooru_v02] add favorite extractor (#1834)

  • [kemonoparty] add favorite extractor (#1824)

  • [kemonoparty] implement login with username & password (#1824)

  • [mastodon] add following extractor (#1891)

  • [mastodon] support specifying accounts by ID

  • [twitter] support /with_replies URLs (#1833)

  • [twitter] add quote_by metadata field (#1481)

  • [postprocessor:compare] extend action option (#1592)

  • implement a download progress indicator (#1519)

  • implement a page-reverse option (#1854)

  • implement a way to specify extended format strings

  • allow specifying a minimum/maximum for sleep-* options (#1835)

  • add a --write-infojson command-line option

Changes

  • [cyberdrop] change directory name format (#1871)

  • [instagram] update default delay to 6-12 seconds (#1835)

  • [reddit] extend subcategory depending on input URL (#1836)

  • move util.Formatter and util.PathFormat into their own modules

Fixes

  • [artstation] use /album/all view for user portfolios (#1826)

  • [aryion] update/improve pagination (#1849)

  • [deviantart] fix bug with fetching premium content (#1879)

  • [deviantart] update default archive_fmt for single deviations (#1874)

  • [erome] send Referer header for file downloads (#1829)

  • [hiperdex] fix extraction

  • [kemonoparty] update file download URLs (#1902, #1903)

  • [mangadex] fix extraction (#1852)

  • [mangadex] fix retrieving chapters from "pornographic" titles (#1908)

  • [nozomi] preserve case of search tags (#1860)

  • [redgifs][gfycat] remove webtoken code (#1907)

  • [twitter] ensure card entries have a url (#1868)

  • implement a way to correctly shorten displayed filenames containing east-asian characters (#1377)

1.18.4 - 2021-09-04

Additions

  • [420chan] add thread and board extractors (#1773)

  • [deviantart] add tag extractor (#1803)

  • [deviantart] add comments option (#1800)

  • [deviantart] implement a auto-watch option (#1466, #1757)

  • [foolfuuka] add gallery extractor (#1785)

  • [furaffinity] expand URL pattern for searches (#1780)

  • [kemonoparty] automatically generate required DDoS-GUARD cookies (#1779)

  • [nhentai] add favorite extractor (#1814)

  • [shopify] support windsorstore.com (#1793)

  • [twitter] add url to user objects (#1787, #1532)

  • [twitter] expand t.co links in user descriptions (#1787, #1532)

  • show a warning if an extractor doesn`t yield any results (#1428, #1759)

  • add a j format string conversion

  • implement a fallback option (#1770)

  • implement a path-strip option

Changes

  • [shopify] use API for product listings (#1793)

  • update default User-Agent headers

Fixes

  • [deviantart] prevent exceptions for "empty" videos (#1796)

  • [exhentai] improve image limits check (#1808)

  • [inkbunny] fix extraction (#1816)

  • [mangadex] prevent exceptions for manga without English title (#1815)

  • [oauth] use defaults when config values are set to null (#1778)

  • [pixiv] fix pixivision title extraction

  • [reddit] delay RedditAPI initialization (#1813)

  • [twitter] improve error reporting (#1759)

  • [twitter] fix issue when filtering quote tweets (#1792)

  • [twitter] fix logout option (#1719)

Removals

  • [deviantart] remove the "you need session cookies to download mature scraps" warning (#1777, #1776)

  • [foolslide] remove entry for kobato.hologfx.com

1.18.3 - 2021-08-13

Additions

  • [bbc] add width option (#1706)

  • [danbooru] add external option (#1747)

  • [furaffinity] add external option (#1492)

  • [luscious] add gif option (#1701)

  • [newgrounds] add format option (#1729)

  • [reactor] add gif option (#1701)

  • [twitter] warn about suspended accounts (#1759)

  • [twitter] extend replies option (#1254)

  • [twitter] add option to log out and retry when blocked (#1719)

  • [wikieat] add thread and board extractors (#1699, #1607)

Changes

  • [instagram] increase default delay between HTTP requests from 5s to 8s (#1732)

Fixes

  • [bbc] improve image dimensions (#1706)

  • [bbc] support multi-page gallery listings (#1730)

  • [behance] fix collection extraction

  • [deviantart] get original files for GIF previews (#1731)

  • [furaffinity] fix errors when using category-transfer (#1274)

  • [hitomi] fix image URLs (#1765)

  • [instagram] use custom User-Agent header for video downloads (#1682, #1623, #1580)

  • [kemonoparty] fix username extraction (#1750)

  • [kemonoparty] update file server domain (#1764)

  • [newgrounds] fix errors when using category-transfer (#1274)

  • [nsfwalbum] retry backend requests when extracting image URLs (#1733, #1271)

  • [vk] prevent exception for empty/private profiles (#1742)

1.18.2 - 2021-07-23

Additions

  • [bbc] add gallery and programme extractors (#1706)

  • [comicvine] add extractor (#1712)

  • [kemonoparty] add max-posts option (#1674)

  • [kemonoparty] parse o query parameters (#1674)

  • [mastodon] add reblogs and replies options (#1669)

  • [pixiv] add extractor for pixivision articles (#1672)

  • [ytdl] add experimental extractor for sites supported by youtube-dl (#1680, #878)

  • extend parent-metadata functionality (#1687, #1651, #1364)

  • add archive-prefix option (#1711)

  • add url-metadata option (#1659, #1073)

Changes

  • [kemonoparty] skip duplicated patreon files (#1689, #1667)

  • [mangadex] use custom User-Agent header (#1535)

Fixes

  • [hitomi] fix image URLs (#1679)

  • [imagevenue] fix extraction (#1677)

  • [instagram] fix extraction of /explore/tags/ posts (#1666)

  • [moebooru] fix tags ending with a + when logged in (#1702)

  • [naverwebtoon] fix comic extraction

  • [pururin] update domain and fix extraction

  • [vk] improve metadata extraction and URL pattern (#1691)

  • [downloader:ytdl] fix outtmpl setting for yt-dlp (#1680)

1.18.1 - 2021-07-04

Additions

  • [mangafox] add manga extractor (#1633)

  • [mangasee] add chapter and manga extractors

  • [mastodon] implement text-posts option (#1569, #1669)

  • [seisoparty] add user and post extractors (#1635)

  • implement conditional directories (#1394)

  • add T format string conversion (#1646)

  • document format string syntax

Changes

  • [twitter] set retweet_id for original retweets (#1481)

Fixes

  • [directlink] manually encode Referer URLs (#1647)

  • [hiperdex] use domain from input URL

  • [kemonoparty] fix username extraction (#1652)

  • [kemonoparty] warn about missing DDoS-GUARD cookies

  • [twitter] ensure guest tokens are returned as string (#1665)

  • [webtoons] match arbitrary language codes (#1643)

  • fix depth counter in UrlJob when specifying -g multiple times

1.18.0 - 2021-06-19

Additions

  • [foolfuuka] support archive.wakarimasen.moe (#1595)

  • [mangadex] implement login with username & password (#1535)

  • [mangadex] add extractor for a user's followed feed (#1535)

  • [pixiv] support fetching privately followed users (#1628)

  • implement conditional filenames (#1394)

  • implement filter option for post processors (#1460)

  • add -T/--terminate command-line option (#1399)

  • add -P/--postprocessor command-line option (#1583)

Changes

  • [kemonoparty] update default filenames and archive IDs (#1514)

  • [twitter] update default settings

    • change retweets and quoted options from true to false

    • change directory format for search results to the same as other extractors

  • require an argument for --clear-cache

Fixes

  • [500px] update GraphQL queries

  • [furaffinity] improve metadata extraction (#1630)

  • [hitomi] update image URL generation (#1637)

  • [idolcomplex] improve and fix pagination (#1594, #1601)

  • [instagram] fix login (#1631)

  • [instagram] update query hashes

  • [mangadex] update to API v5 (#1535)

  • [mangafox] improve URL pattern (#1608)

  • [oauth] prevent exceptions when reporting errors (#1603)

  • [philomena] fix tag escapes handling (#1629)

  • [redgifs] update API server address (#1632)

  • [sankaku] handle empty tags (#1617)

  • [subscribestar] improve attachment filenames (#1609)

  • [unsplash] update collections URL pattern (#1627)

  • [postprocessor:metadata] handle dicts in mode:tags (#1598)

1.17.5 - 2021-05-30

Additions

  • [kemonoparty] add metadata option (#1548)

  • [kemonoparty] add type metadata field (#1556)

  • [mangapark] recognize v2.mangapark URLs (#1578)

  • [patreon] extract user-defined tags (#1539, #1540)

  • [pillowfort] implement login with username & password (#846)

  • [pillowfort] add inline and external options (#846)

  • [pixiv] implement max-posts option (#1558)

  • [pixiv] add metadata option (#1551)

  • [twitter] add text-tweets option (#570)

  • [weibo] extend retweets option (#1542)

  • [postprocessor:ugoira] support using the image2 demuxer (#1550)

  • [postprocessor:ugoira] add repeat-last-frame option (#1550)

  • support XDG_CONFIG_HOME (#1545)

  • implement parent-skip and "skip": "terminate" options (#1399)

Changes

  • [twitter] resolve t.co URLs in content (#1532)

Fixes

  • [500px] update query hashes (#1573)

  • [aryion] find text posts in recursive=false mode (#1568)

  • [imagebam] fix extraction of NSFW images (#1534)

  • [imgur] update URL patterns (#1561)

  • [manganelo] update domain to manganato.com

  • [reactor] skip deleted/empty posts

  • [twitter] add missing retweet media entities (#1555)

  • fix ISO 639-1 code for Japanese (jp -> ja)

1.17.4 - 2021-05-07

Additions

  • [gelbooru] add extractor for /redirect.php URLs (#1530)

  • [inkbunny] add favorite extractor (#1521)

  • add output.skip option

  • add an optional argument to --clear-cache to select which cache entries to remove (#1230)

Changes

  • [pixiv] update translated-tags option (#1507)

    • rename to tags

    • accept "japanese", "translated", and "original" as values

Fixes

  • [500px] update query hashes

  • [kemonoparty] fix download URLs (#1514)

  • [imagebam] fix extraction

  • [instagram] update query hashes

  • [nozomi] update default archive-fmt for tag and search extractors (#1529)

  • [pixiv] remove duplicate translated tags (#1507)

  • [readcomiconline] change domain to readcomiconline.li (#1517)

  • [sankaku] update invalid-token detection (#1515)

  • fix crash when using --no-download with --ugoira-conv (#1507)

1.17.3 - 2021-04-25

Additions

  • [danbooru] add option for extended metadata extraction (#1458)

  • [fanbox] add extractors (#1459)

  • [fantia] add extractors (#1459)

  • [gelbooru] add an option to extract notes (#1457)

  • [hentaicosplays] add extractor (#907, #1473, #1483)

  • [instagram] add extractor for tagged posts (#1439)

  • [naverwebtoon] ignore non-comic images

  • [pixiv] also save untranslated tags when translated-tags is enabled (#1501)

  • [shopify] support omgmiamiswimwear.com (#1280)

  • implement output.fallback option

  • add archive format to InfoJob output (#875)

  • build executables with SOCKS proxy support (#1424)

Fixes

  • [500px] update query hashes

  • [8muses] fix JSON deobfuscation

  • [artstation] download /4k/ images (#1422)

  • [deviantart] fix pagination for Eclipse results (#1444)

  • [deviantart] improve folder name matching (#1451)

  • [erome] skip deleted albums (#1447)

  • [exhentai] fix image limit detection (#1437)

  • [exhentai] restore limits option (#1487)

  • [gelbooru] fix tag category extraction (#1455)

  • [instagram] update query hashes

  • [komikcast] fix extraction

  • [simplyhentai] fix extraction

  • [slideshare] fix extraction

  • [webtoons] update agegate/GDPR cookies (#1431)

  • fix category-transfer option

Removals

1.17.2 - 2021-04-02

Additions

  • [deviantart] add support for posts from watched users (#794)

  • [manganelo] add chapter and manga extractors (#1415)

  • [pinterest] add search extractor (#1411)

  • [sankaku] add tag_string metadata field (#1388)

  • [sankaku] add enumeration index for books (#1388)

  • [tapas] add series and episode extractors (#692)

  • [tapas] implement login with username & password (#692)

  • [twitter] allow specifying a custom format for user results (#1337)

  • [twitter] add extractor for direct image links (#1417)

  • [vk] add support for albums (#474)

Fixes

  • [aryion] unescape paths (#1414)

  • [bcy] improve pagination

  • [deviantart] update watch URL pattern (#794)

  • [deviantart] fix arguments for search/popular results (#1408)

  • [deviantart] use fallback for /intermediary/ URLs

  • [exhentai] improve and simplify image limit checks

  • [komikcast] fix extraction

  • [pixiv] fix favorite URL pattern (#1405)

  • [sankaku] simplify pool tags (#1388)

  • [twitter] improve error message when trying to log in with 2FA (#1409)

  • [twitter] don't use youtube-dl for cards when videos are disabled (#1416)

1.17.1 - 2021-03-19

Additions

  • [architizer] add project and firm extractors (#1369)

  • [deviantart] add watch extractor (#794)

  • [exhentai] support /tag/ URLs (#1363)

  • [gelbooru_v01] support drawfriends.booru.org, vidyart.booru.org, and tlb.booru.org by default

  • [nozomi] support /index-N.html URLs (#1365)

  • [philomena] add generalized extractors for philomena sites (#1379)

  • [philomena] support post URLs without /images/

  • [twitter] implement users option (#1337)

  • implement parent-metadata option (#1364)

Changes

  • [deviantart] revert previous changes to extra option (#1356, #1387)

Fixes

  • [exhentai] improve favorites count extraction (#1360)

  • [gelbooru] update domain for video downloads (#1368)

  • [hentaifox] improve image and metadata extraction (#1366, #1378)

  • [imgur] fix and improve rate limit handling (#1386)

  • [weasyl] improve favorites URL pattern (#1374)

  • use type check before applying browser option (#1358)

  • ensure -s/--simulate always prints filenames (#1360)

Removals

  • [hentaicafe] remove module

  • [hentainexus] remove module

  • [mangareader] remove module

  • [mangastream] remove module

1.17.0 - 2021-03-05

Additions

  • [cyberdrop] add support for https://cyberdrop.me/ (#1328)

  • [exhentai] add metadata option; extract more metadata from gallery pages (#1325)

  • [hentaicafe] add search and tag extractors (#1345)

  • [hentainexus] add original option (#1322)

  • [instagram] support /user/reels/ URLs (#1329)

  • [naverwebtoon] add support for https://comic.naver.com/ (#1331)

  • [pixiv] add translated-tags option (#1354)

  • [tbib] add support for https://tbib.org/ (#473, #1082)

  • [tumblrgallery] add support for https://tumblrgallery.xyz/ (#1298)

  • [twitter] add extractor for followed users (#1337)

  • [twitter] add option to download all media from conversations (#1319)

  • [wallhaven] add collections extractor (#1351)

  • [snap] allow access to user's .netrc for site authentication (#1352)

  • add extractors for Gelbooru v0.1 sites (#234, #426, #473, #767, #1238)

  • add -E/--extractor-info command-line option (#875)

  • add GitHub Actions workflow for building standalone executables (#1312)

  • add browser and headers options (#1117)

  • add option to use different youtube-dl forks (#1330)

  • support using multiple input files at once (#1353)

Changes

  • [deviantart] extend extra option to also download embedded DeviantArt posts.

  • [exhentai] rename metadata fields to match API results (#1325)

  • [mangadex] use api.mangadex.org as default API server

  • [mastodon] cache OAuth tokens (#616)

  • replace wait-min and wait-max with sleep-request

Fixes

  • [500px] skip unavailable photos (#1335)

  • [komikcast] fix extraction

  • [readcomiconline] download high quality image versions (#1347)

  • [twitter] update GraphQL endpoints

  • fix crash when base-directory is an empty string (#1339)

Removals

  • remove support for formerly deprecated options

  • remove cloudflare module

1.16.5 - 2021-02-14

Additions

  • [behance] support video modules (#1282)

  • [erome] add album, user, and search extractors (#409)

  • [hentaifox] support searching by group (#1294)

  • [imgclick] add image extractor (#1307)

  • [kemonoparty] extract inline images (#1286)

  • [kemonoparty] support URLs with non-numeric user and post IDs (#1303)

  • [pillowfort] add user and post extractors (#846)

Changes

  • [kemonoparty] include service in directories and archive keys

  • [pixiv] require a refresh-token to login (#1304)

  • [snap] use core18 as base

Fixes

  • [500px] update query hashes

  • [deviantart] update parameters for /browse/popular (#1267)

  • [deviantart] provide filename extension for original file downloads (#1272)

  • [deviantart] fix folders option (#1302)

  • [inkbunny] add sid parameter to private file downloads (#1281)

  • [kemonoparty] fix absolute file URLs

  • [mangadex] revert to https://mangadex.org/api/ and add api-server option (#1310)

  • [nsfwalbum] use fallback for deleted content (#1259)

  • [sankaku] update invalid token detection (#1309)

  • [slideshare] fix extraction

  • [postprocessor:metadata] fix crash with extension-format (#1285)

1.16.4 - 2021-01-23

Additions

  • [furaffinity] add descriptions option (#1231)

  • [kemonoparty] add user and post extractors (#1216)

  • [nozomi] add num enumeration index (#1239)

  • [photovogue] added portfolio extractor (#1253)

  • [twitter] match /i/user/ID URLs

  • [unsplash] add extractors (#1197)

  • [vipr] add image extractor (#1258)

Changes

  • [derpibooru] use "Everything" filter by default (#862)

Fixes

  • [derpibooru] update date parsing

  • [foolfuuka] stop search when results are exhausted (#1174)

  • [instagram] fix regex for /saved URLs (#1251)

  • [mangadex] update API URLs

  • [mangakakalot] fix extraction

  • [newgrounds] fix flash file extraction (#1257)

  • [sankaku] simplify login process

  • [twitter] fix retries after hitting rate limit

1.16.3 - 2021-01-10

Fixes

  • fix crash when using a dict for path-restrict

  • [postprocessor:metadata] sanitize custom filenames

1.16.2 - 2021-01-09

Additions

  • [derpibooru] add search and gallery extractors (#862)

  • [foolfuuka] add board and search extractors (#1044, #1174)

  • [gfycat] add date metadata field (#1138)

  • [pinterest] add support for getting all boards of a user (#1205)

  • [sankaku] add support for book searches (#1204)

  • [twitter] fetch media from pinned tweets (#1203)

  • [wikiart] add extractor for single paintings (#1233)

  • [downloader:http] add MIME type and signature for .ico files (#1211)

  • add d format string conversion for timestamp values

  • add "ascii" as a special path-restrict value

Fixes

  • [hentainexus] fix extraction (#1234)

  • [instagram] categorize single highlight URLs as highlights (#1222)

  • [redgifs] fix search results

  • [twitter] fix login with username & password

  • [twitter] fetch tweets from homeConversation entries

1.16.1 - 2020-12-27

Additions

  • [instagram] add include option (#1180)

  • [pinterest] implement video support (#1189)

  • [sankaku] reimplement login support (#1176, #1182)

  • [sankaku] add support for sankaku.app URLs (#1193)

Changes

  • [e621] return pool posts in order (#1195)

  • [hentaicafe] prefer title of /hc.fyi/ pages (#1106)

  • [hentaicafe] simplify default filenames

  • [sankaku] normalize created_at metadata (#1190)

  • [postprocessor:exec] do not add missing {} to command (#1185)

Fixes

  • [booru] improve error handling

  • [instagram] warn about private profiles (#1187)

  • [keenspot] improve redirect handling

  • [mangadex] respect chapter-reverse settings (#1194)

  • [pixiv] output debug message on failed login attempts (#1192)

  • increase SQLite connection timeouts (#1173)

Removals

  • [mangapanda] remove module

1.16.0 - 2020-12-12

Additions

  • [booru] implement generalized extractors for *booru and moebooru sites

  • [postprocessor:metadata] add event and filename options (#315, #866, #984)

  • [postprocessor:exec] add event option (#992)

Changes

  • [flickr] update default directories and improve metadata consistency (#828)

  • [sankaku] use API endpoints from beta.sankakucomplex.com

  • [downloader:http] improve filename extension handling (#776)

  • replace all JPEG filename extensions with jpg by default

Fixes

  • [hentainexus] fix extraction (#1166)

  • [instagram] rewrite (#1113, #1122, #1128, #1130, #1149)

  • [mangadex] handle external chapters (#1154)

  • [nozomi] handle empty date fields (#1163)

  • [paheal] create directory for each post (#1147)

  • [piczel] update API URLs

  • [twitter] update image URL format (#1145)

  • [twitter] improve x-csrf-token header handling (#1170)

  • [webtoons] update ageGate cookies

Removals

  • [sankaku] remove login support

1.15.4 - 2020-11-27

Fixes

  • [2chan] skip external links

  • [hentainexus] fix extraction (#1125)

  • [mangadex] switch to API v2 (#1129)

  • [mangapanda] use http://

  • [mangoxo] fix extraction

  • [reddit] skip invalid gallery items (#1127)

1.15.3 - 2020-11-13

Additions

  • [sankakucomplex] extract videos and embeds (#308)

  • [twitter] add support for lists (#1096)

  • [postprocessor:metadata] accept string-lists for content-format (#1080)

  • implement modules and extension-map options

Fixes

  • [500px] update query hashes

  • [8kun] fix file URLs of older posts (#1101)

  • [exhentai] update image URL parsing (#1094)

  • [hentaifoundry] update YII_CSRF_TOKEN cookie handling (#1083)

  • [hentaifoundry] use scheme from input URLs (#1095)

  • [mangoxo] fix metadata extraction

  • [paheal] fix extraction (#1088)

  • collect post processors from basecategory entries (#1084)

1.15.2 - 2020-10-24

Additions

  • [pinterest] implement login support (#1055)

  • [reddit] add date metadata field (#1068)

  • [seiga] add metadata for single image downloads (#1063)

  • [twitter] support media from Cards (#937, #1005)

  • [weasyl] support api-key authentication (#1057)

  • add a t format string conversion for trimming whitespace (#1065)

Fixes

  • [blogger] handle URLs with specified width/height (#1061)

  • [fallenangels] fix extraction of .5 chapters

  • [gelbooru] rewrite mp4 video URLs (#1048)

  • [hitomi] fix image URLs and gallery URL pattern

  • [mangadex] unescape more metadata fields (#1066)

  • [mangahere] ensure download URLs have a scheme (#1070)

  • [mangakakalot] ignore "Go Home" buttons in chapter pages

  • [newgrounds] handle embeds without scheme (#1033)

  • [newgrounds] provide fallback URLs for video downloads (#1042)

  • [xhamster] fix user profile extraction

1.15.1 - 2020-10-11

Additions

  • [hentaicafe] add manga_id metadata field (#1036)

  • [hentaifoundry] add support for stories (#734)

  • [hentaifoundry] add include option

  • [newgrounds] extract image embeds (#1033)

  • [nijie] add include option (#1018)

  • [reactor] match URLs without subdomain (#1053)

  • [twitter] extend retweets option (#1026)

  • [weasyl] add extractors (#977)

Fixes

  • [500px] update query hashes

  • [behance] fix collection extraction

  • [newgrounds] fix video extraction (#1042)

  • [twitter] improve twitpic extraction (#1019)

  • [weibo] handle posts with more than 9 images (#926)

  • [xvideos] fix title extraction

  • fix crash when using --download-archive with --no-skip (#1023)

  • fix issues with blacklist/whitelist defaults (#1051, #1056)

Removals

  • [kissmanga] remove module

1.15.0 - 2020-09-20

Additions

  • [deviantart] support watchers-only/paid deviations (#995)

  • [myhentaigallery] add gallery extractor (#1001)

  • [twitter] support specifying users by ID (#980)

  • [twitter] support /intent/user?user_id=… URLs (#980)

  • add --no-skip command-line option (#986)

  • add blacklist and whitelist options (#492, #844)

  • add filesize-min and filesize-max options (#780)

  • add sleep-extractor and sleep-request options (#788)

  • write skipped files to archive (#550)

Changes

  • [exhentai] update wait time before original image downloads (#978)

  • [imgur] use new API endpoints for image/album data

  • [tumblr] create directories for each post (#965)

  • support format string replacement fields in download archive paths (#985)

  • reduce wait time growth rate for HTTP retries from exponential to linear

Fixes

  • [500px] update query hash

  • [aryion] improve post ID extraction (#981, #982)

  • [danbooru] handle posts without id (#1004)

  • [furaffinity] update download URL extraction (#988)

  • [imgur] fix image/album detection for galleries

  • [postprocessor:zip] defer zip file creation (#968)

Removals

  • [jaiminisbox] remove extractors

  • [worldthree] remove extractors

1.14.5 - 2020-08-30

Additions

  • [aryion] add username/password support (#960)

  • [exhentai] add ability to specify a custom image limit (#940)

  • [furaffinity] add search extractor (#915)

  • [imgur] add search and tag extractors (#934)

Fixes

  • [500px] fix extraction and update URL patterns (#956)

  • [aryion] update folder mime type list (#945)

  • [gelbooru] fix extraction without API

  • [hentaihand] update to new site layout

  • [hitomi] fix redirect processing

  • [reddit] handle deleted galleries (#953)

  • [reddit] improve gallery extraction (#955)

1.14.4 - 2020-08-15

Additions

  • [blogger] add search extractor (#925)

  • [blogger] support searching posts by labels (#925)

  • [inkbunny] add user and post extractors (#283)

  • [instagram] support /reel/ URLs

  • [pinterest] support pinterest.co.uk URLs (#914)

  • [reddit] support gallery posts (#920)

  • [subscribestar] extract attached media files (#852)

Fixes

  • [blogger] improve error messages for missing posts/blogs (#903)

  • [exhentai] adjust image limit costs (#940)

  • [gfycat] skip malformed gfycat responses (#902)

  • [imgur] handle 403 overcapacity responses (#910)

  • [instagram] wait before GraphQL requests (#901)

  • [mangareader] fix extraction

  • [mangoxo] fix login

  • [pixnet] detect password-protected albums (#177)

  • [simplyhentai] fix gallery_id extraction

  • [subscribestar] update date parsing

  • [vsco] handle missing description fields

  • [xhamster] fix extraction (#917)

  • allow parent-directory to work recursively (#905)

  • skip external OAuth tests (#908)

Removals

  • [bobx] remove module

1.14.3 - 2020-07-18

Additions

  • [8muses] support comics.8muses.com URLs

  • [artstation] add following extractor (#888)

  • [exhentai] add domain option (#897)

  • [gfycat] add user and search extractors

  • [imgur] support all /t/... URLs (#880)

  • [khinsider] add format option (#840)

  • [mangakakalot] add manga and chapter extractors (#876)

  • [redgifs] support gifsdeliverynetwork.com URLs (#874)

  • [subscribestar] add user and post extractors (#852)

  • [twitter] add support for nitter.net URLs (#890)

  • add Zsh completion script (#150)

Fixes

  • [gfycat] retry 404'ed videos on redgifs.com (#874)

  • [newgrounds] fix favorites extraction

  • [patreon] yield images and attachments before post files (#871)

  • [reddit] fix AttributeError when using recursion (#879)

  • [twitter] raise proper exception if a user doesn't exist (#891)

  • defer directory creation (#722)

  • set pseudo extension for Metadata messages (#865)

  • prevent exception on Cloudflare challenges (#868)

1.14.2 - 2020-06-27

Additions

  • [artstation] add date metadata field (#839)

  • [mastodon] add date metadata field (#839)

  • [pinterest] add support for board sections (#835)

  • [twitter] add extractor for liked tweets (#837)

  • [twitter] add option to filter media from quoted tweets (#854)

  • [weibo] add date metadata field to status objects (#829)

Fixes

  • [aryion] fix user gallery extraction (#832)

  • [imgur] build directory paths for each file (#842)

  • [tumblr] prevent errors when using reblogs=same-blog (#851)

  • [twitter] always provide an author metadata field (#831, #833)

  • [twitter] don't download video previews (#833)

  • [twitter] improve handling of deleted tweets (#838)

  • [twitter] fix search results (#847)

  • [twitter] improve handling of quoted tweets (#854)

  • fix config lookups when multiple locations are involved (#843)

  • improve output of -K/--list-keywords for parent extractors (#825)

  • call flush() after writing JSON in DataJob() (#727)

1.14.1 - 2020-06-12

Additions

  • [furaffinity] add artist_url metadata field (#821)

  • [redgifs] add user and search extractors (#724)

Changes

  • [deviantart] extend extra option; also search journals for sta.sh links (#712)

  • [twitter] rewrite; use new interface (#806, #740)

Fixes

  • [kissmanga] work around CAPTCHAs (#818)

  • [nhentai] fix extraction (#819)

  • [webtoons] generalize comic extraction code (#820)

1.14.0 - 2020-05-31

Additions

  • [imagechest] add new extractor for imgchest.com (#750)

  • [instagram] add post_url, tags, location, tagged_users metadata (#743)

  • [redgifs] add image extractor (#724)

  • [webtoons] add new extractor for webtoons.com (#761)

  • implement --write-pages option (#736)

  • extend path-restrict option (#662)

  • implement path-replace option (#662, #755)

  • make path and keywords available in logging messages (#574, #575)

Changes

  • [danbooru] change default value of ugoira to false

  • [downloader:ytdl] change default value of forward-cookies to false

  • [downloader:ytdl] fix file extensions when merging into .mkv (#720)

  • write OAuth tokens to cache (#616)

  • use %APPDATA%\gallery-dl for config files and cache on Windows

  • use util.Formatter for formatting logging messages

  • reuse HTTP connections from parent extractors

Fixes

  • [deviantart] use private access tokens for Journals (#738)

  • [gelbooru] simplify and fix pool extraction

  • [imgur] fix extraction of animated images without mp4 entry

  • [imgur] treat /t/unmuted/ URLs as galleries

  • [instagram] fix login with username & password (#756, #771, #797, #803)

  • [reddit] don't send OAuth headers for file downloads (#729)

  • fix/improve Cloudflare bypass code (#728, #757)

  • reset filenames on empty file extensions (#733)

1.13.6 - 2020-05-02

Additions

  • [patreon] respect filters and sort order in query parameters (#711)

  • [speakerdeck] add a new extractor for speakerdeck.com (#726)

  • [twitter] add replies option (#705)

  • [weibo] add videos option

  • [downloader:http] add MIME types for .psd files (#714)

Fixes

  • [artstation] improve embed extraction (#720)

  • [deviantart] limit API wait times (#721)

  • [newgrounds] fix URLs produced by the following extractor (#684)

  • [patreon] improve file hash extraction (#713)

  • [vsco] fix user gallery extraction

  • fix/improve Cloudflare bypass code (#728)

1.13.5 - 2020-04-27

Additions

  • [500px] recognize web.500px.com URLs

  • [aryion] support downloading from folders (#694)

  • [furaffinity] add extractor for followed users (#515)

  • [hitomi] add extractor for tag searches (#697)

  • [instagram] add post_id and num metadata fields (#698)

  • [newgrounds] add extractor for followed users (#684)

  • [patreon] recognize URLs with creator IDs (#711)

  • [twitter] add reply metadata field (#705)

  • [xhamster] recognize xhamster.porncache.net URLs (#700)

Fixes

  • [gelbooru] improve post ID extraction in pool listings

  • [hitomi] fix extraction of galleries without tags

  • [jaiminisbox] update metadata decoding procedure (#702)

  • [mastodon] fix pagination (#701)

  • [mastodon] improve account searches (#704)

  • [patreon] fix hash extraction from download URLs (#693)

  • improve parameter extraction when solving Cloudflare challenges

1.13.4 - 2020-04-12

Additions

  • [aryion] add gallery and post extractors (#390, #673)

  • [deviantart] detect and handle folders in sta.sh listings (#659)

  • [hentainexus] add circle, event, and title_conventional metadata fields (#661)

  • [hiperdex] add artist extractor (#606)

  • [mastodon] add access tokens for mastodon.social and baraag.net (#665)

Changes

  • [deviantart] retrieve all download URLs through the OAuth API

  • automatically read config files in PyInstaller executable directories (#682)

Fixes

  • [deviantart] handle "Request blocked" errors (#655)

  • [deviantart] improve JPEG quality replacement pattern

  • [hiperdex] fix extraction

  • [mastodon] handle API rate limits (#665)

  • [mastodon] update OAuth credentials for pawoo.net (#665)

  • [myportfolio] fix extraction of galleries without title

  • [piczel] fix extraction of single images

  • [vsco] fix collection extraction

  • [weibo] accept status URLs with non-numeric IDs (#664)

1.13.3 - 2020-03-28

Additions

  • [instagram] Add support for user's saved medias (#644)

  • [nozomi] support multiple images per post (#646)

  • [35photo] add tag extractor

Changes

  • [mangadex] transform timestamps from date fields to datetime objects

Fixes

  • [deviantart] handle decode errors for extended_fetch results (#655)

  • [e621] fix bug in API rate limiting and improve pagination (#651)

  • [instagram] update pattern for user profile URLs

  • [mangapark] fix metadata extraction

  • [nozomi] sort search results (#646)

  • [piczel] fix extraction

  • [twitter] fix typo in x-twitter-auth-type header (#625)

  • remove trailing dots from Windows directory names (#647)

  • fix crash with missing stdout/stderr/stdin handles (#653)

1.13.2 - 2020-03-14

Additions

  • [furaffinity] extract more metadata

  • [instagram] add post_shortcode metadata field (#525)

  • [kabeuchi] add extractor (#561)

  • [newgrounds] add extractor for favorited posts (#394)

  • [pixiv] implement avatar option (#595, #623)

  • [twitter] add extractor for bookmarked Tweets (#625)

Fixes

  • [bcy] reduce number of HTTP requests during data extraction

  • [e621] update to new interface (#635)

  • [exhentai] handle incomplete MIME types (#632)

  • [hitomi] improve metadata extraction

  • [mangoxo] fix login

  • [newgrounds] improve error handling when extracting post data

1.13.1 - 2020-03-01

Additions

  • [hentaihand] add extractors (#605)

  • [hiperdex] add chapter and manga extractors (#606)

  • [oauth] implement option to write DeviantArt refresh-tokens to cache (#616)

  • [downloader:http] add more MIME types for .bmp and .rar files (#621, #628)

  • warn about expired cookies

Fixes

  • [bcy] fix partial image URLs (#613)

  • [danbooru] fix Ugoira downloads and metadata

  • [deviantart] check availability of /intermediary/ URLs (#609)

  • [hitomi] follow multiple redirects & fix image URLs

  • [piczel] improve and update

  • [tumblr] replace - with in tag searches (#611)

  • [vsco] update gallery URL pattern

  • fix --verbose and --quiet command-line options

1.13.0 - 2020-02-16

Additions

  • Support for

  • [blogger] implement video extraction (#587)

  • [oauth] add option to specify port number used by local server (#604)

  • [pixiv] add rating metadata field (#595)

  • [pixiv] recognize tags at the end of new bookmark URLs

  • [reddit] add videos option

  • [weibo] use youtube-dl to download from m3u8 manifests

  • implement parent-directory option (#551)

  • extend filename formatting capabilities:

    • implement field name alternatives (#525)

    • allow multiple "special" format specifiers per replacement field (#595)

    • allow for numeric list and string indices

Changes

  • [reddit] handle reddit-hosted images and videos natively (#551)

  • [twitter] change default value for videos to true

Fixes

  • [cloudflare] unescape challenge URLs

  • [deviantart] fix video extraction from extended_fetch results

  • [hitomi] implement workaround for "broken" redirects

  • [khinsider] fix and improve metadata extraction

  • [patreon] filter duplicate files per post (#590)

  • [piczel] fix extraction

  • [pixiv] fix user IDs for bookmarks API calls (#596)

  • [sexcom] fix image URLs

  • [twitter] force old login page layout (#584, #598)

  • [vsco] skip "invalid" entities

  • improve functions to load/save cookies.txt files (#586)

Removals

  • [yaplog] remove module

1.12.3 - 2020-01-19

Additions

  • [hentaifoundry] extract more metadata (#565)

  • [twitter] add option to extract TwitPic embeds (#579)

  • implement a post-processor module to compare file versions (#530)

Fixes

  • [hitomi] update image URL generation

  • [mangadex] revert domain to mangadex.org

  • [pinterest] improve detection of invalid pin.it links

  • [pixiv] update URL patterns for user profiles and bookmarks (#568)

  • [twitter] Fix stop before real end (#573)

  • remove temp files before downloading from fallback URLs

Removals

  • [erolord] remove extractor

1.12.2 - 2020-01-05

Additions

  • [deviantart] match new search/popular URLs (#538)

  • [deviantart] match /favourites/all URLs (#555)

  • [deviantart] add extractor for followed users (#515)

  • [pixiv] support listing followed users (#515)

  • [imagefap] handle beta.imagefap.com URLs (#552)

  • [postprocessor:metadata] add directory option (#520)

Fixes

  • [artstation] fix search result pagination (#537)

  • [directlink] send Referer headers (#536)

  • [exhentai] restrict default directory name length (#545)

  • [mangadex] change domain to mangadex.cc (#559)

  • [mangahere] send isAdult cookies (#556)

  • [newgrounds] fix tags metadata extraction

  • [pixiv] retry after rate limit errors (#535)

  • [twitter] handle quoted tweets (#526)

  • [twitter] handle API rate limits (#526)

  • [twitter] fix URLs forwarded to youtube-dl (#540)

  • prevent infinite recursion when spawning new extractors (#489)

  • improve output of --list-keywords for "parent" extractors (#548)

  • provide fallback for SQLite versions with missing WITHOUT ROWID support (#553)

1.12.1 - 2019-12-22

Additions

  • [4chan] add extractor for entire boards (#510)

  • [realbooru] add extractors for pools, posts, and tag searches (#514)

  • [instagram] implement a videos option (#521)

  • [vsco] implement a videos option

  • [postprocessor:metadata] implement a bypost option for downloading the metadata of an entire post (#511)

Changes

  • [reddit] change the default value for comments to 0

  • [vsco] improve image resolutions

  • make filesystem-related errors during file downloads non-fatal (#512)

Fixes

  • [foolslide] add fallback for chapter data extraction

  • [instagram] ignore errors during post-page extraction

  • [patreon] avoid errors when fetching user info (#508)

  • [patreon] improve URL pattern for single posts

  • [reddit] fix errors with t1 submissions

  • [vsco] fix user profile extraction … again

  • [weibo] handle unavailable/deleted statuses

  • [downloader:http] improve rate limit handling

  • retain trailing zeroes in Cloudflare challenge answers

1.12.0 - 2019-12-08

Additions

  • [flickr] support 3k, 4k, 5k, and 6k photo sizes (#472)

  • [imgur] add extractor for subreddit links (#500)

  • [newgrounds] add extractors for audio listings and general media files (#394)

  • [newgrounds] implement login support (#394)

  • [postprocessor:metadata] implement a extension-format option (#477)

  • --exec-after

Changes

  • [deviantart] ensure consistent username capitalization (#455)

  • [directlink] split {path} into {path}/{filename}.{extension}

  • [twitter] update metadata fields with user/author information

  • [postprocessor:metadata] filter private entries & rename format to content-format

  • Enable cookies-update by default

Fixes

  • [2chan] fix metadata extraction

  • [behance] get images from 'media_collection' modules

  • [bobx] fix image downloads by randomly generating session cookies (#482)

  • [deviantart] revert to getting download URLs from OAuth API calls (#488)

  • [deviantart] fix URL generation from '/extended_fetch' results (#505)

  • [flickr] adjust OAuth redirect URI (#503)

  • [hentaifox] fix extraction

  • [imagefap] adapt to new image URL format

  • [imgbb] fix error in galleries without user info (#471)

  • [instagram] prevent errors with missing 'video_url' fields (#479)

  • [nijie] fix date parsing

  • [pixiv] match new search URLs (#507)

  • [plurk] fix comment pagination

  • [sexcom] send specific Referer headers when downloading videos

  • [twitter] fix infinite loops (#499)

  • [vsco] fix user profile and collection extraction (#480)

  • Fix Cloudflare DDoS protection bypass

Removals

  • --abort-on-skip

1.11.1 - 2019-11-09

Fixes

  • Fix inclusion of bash completion and man pages in source distributions

1.11.0 - 2019-11-08

Additions

Changes

  • Rewrite imgur using its public API (#446)

  • Rewrite luscious using GraphQL queries (#457)

  • Adjust default nijie filenames to match pixiv

  • Change enumeration index for gallery extractors from page to num

  • Return non-zero exit status when errors occurred

  • Forward proxy settings to youtube-dl downloader

  • Install bash completion script into share/bash-completion/completions

Fixes

  • Adapt to new instagram page layout when logged in (#391)

  • Support protected twitter videos (#452)

  • Extend hitomi URL pattern and fix gallery extraction

  • Restore OAuth2 authentication error messages

  • Miscellaneous fixes for patreon (#444), deviantart (#455), sexcom (#464), imgur (#467), simplyhentai

1.10.6 - 2019-10-11

Additions

  • --exec command-line option to specify a command to run after each file download (#421)

Changes

  • Include titles in gfycat default filenames (#434)

Fixes

  • Fetch working download URLs for deviantart (#436)

  • Various fixes and improvements for yaplog blogs (#443)

  • Fix image URL generation for hitomi galleries

  • Miscellaneous fixes for behance and xvideos

1.10.5 - 2019-09-28

Additions

  • instagram.highlights option to include highlighted stories when downloading user profiles (#329)

  • Support for /user/ URLs on reddit (#350)

  • Support for imgur user profiles and favorites (#420)

  • Additional metadata fields on nijie(#423)

Fixes

  • Improve handling of private deviantart artworks (#414) and 429 status codes (#424)

  • Prevent fatal errors when trying to open download-archive files (#417)

  • Detect and ignore unavailable videos on weibo (#427)

  • Update the scope of new reddit refresh-tokens (#428)

  • Fix inconsistencies with the reddit.comments option (#429)

  • Extend URL patterns for hentaicafe manga and pixiv artworks

  • Improve detection of unavailable albums on luscious and imgbb

  • Miscellaneous fixes for tsumino

1.10.4 - 2019-09-08

Additions

Fixes

  • Download the correct files from twitter replies (#403)

  • Prevent crash when trying to use unavailable downloader modules (#405)

  • Fix pixiv authentication (#411)

  • Improve exhentai image limit checks

  • Miscellaneous fixes for hentaicafe, simplyhentai, tumblr

1.10.3 - 2019-08-30

Additions

  • Provide filename metadata for all deviantart files (#392, #400)

  • Implement a ytdl.outtmpl option to let youtube-dl handle filenames by itself (#395)

  • Support seiga mobile URLs (#401)

Fixes

  • Extract more than the first 32 posts from piczel galleries (#396)

  • Fix filenames of archives created with --zip (#397)

  • Skip unavailable images and videos on flickr (#398)

  • Fix filesystem paths on Windows with Python 3.6 and lower (#402)

1.10.2 - 2019-08-23

Additions

  • Support for instagram stories and IGTV (#371, #373)

  • Support for individual imgbb images (#363)

  • deviantart.quality option to set the JPEG compression quality for newer images (#369)

  • enumerate option for extractor.skip (#306)

  • adjust-extensions option to control filename extension adjustments

  • path-remove option to remove control characters etc. from filesystem paths

Changes

  • Rename restrict-filenames to path-restrict

  • Adjust pixiv metadata and default filename format (#366)

    • Set filename to "{category}_{user[id]}_{id}{suffix}.{extension}" to restore the old default

  • Improve and optimize directory and filename generation

Fixes

  • Allow the classify post-processor to handle files with unknown filename extension (#138)

  • Fix rate limit handling for OAuth APIs (#368)

  • Fix artwork and scraps extraction on deviantart (#376, #392)

  • Distinguish between imgur album and gallery URLs (#380)

  • Prevent crash when using --ugoira-conv (#382)

  • Handle multi-image posts on patreon (#383)

  • Miscellaneous fixes for *reactor, simplyhentai

1.10.1 - 2019-08-02

Fixes

1.10.0 - 2019-08-01

Warning

  • Prior to version 1.10.0 all cache files were created world readable (mode 644) leading to possible sensitive information disclosure on multi-user systems

  • It is recommended to restrict access permissions of already existing files (/tmp/.gallery-dl.cache) with chmod 600

  • Windows users should not be affected

Additions

Changes

  • The default cache file location on non-Windows systems is now

    • $XDG_CACHE_HOME/gallery-dl/cache.sqlite3 or

    • ~/.cache/gallery-dl/cache.sqlite3

  • New cache files are created with mode 600

  • exhentai extractors will always use e-hentai.org as domain

Fixes

  • Better handling of exhentai image limits and errors (#356, #360)

  • Try to prevent ZIP file corruption (#355)

  • Miscellaneous fixes for behance, ngomik

1.9.0 - 2019-07-19

Additions

  • Support for

  • Add login support for instagram (#195)

  • Add --no-download and extractor.*.download disable file downloads (#220)

  • Add -A/--abort to specify the number of consecutive download skips before aborting

  • Interpret -1 as infinite retries (#300)

  • Implement custom log message formats per log-level (#304)

  • Implement an mtime post-processor that sets file modification times according to metadata fields (#332)

  • Implement a twitter.content option to enable tweet text extraction (#333, #338)

  • Enable date-min/-max/-format options for tumblr (#337)

Changes

  • Set file modification times according to their Last-Modified header when downloading (#236, #277)

    • Use --no-mtime or downloader.*.mtime to disable this behavior

  • Duplicate download URLs are no longer silently ignored (controllable with extractor.*.image-unique)

  • Deprecate --abort-on-skip

Fixes

  • Retry downloads on OpenSSL exceptions (#324)

  • Ignore unavailable pins on sexcom instead of raising an exception (#325)

  • Use Firefox's SSL/TLS ciphers to prevent Cloudflare CAPTCHAs (#342)

  • Improve folder name matching on deviantart (#343)

  • Forward cookies to youtube-dl to allow downloading private videos

  • Miscellaneous fixes for 35photo, 500px, newgrounds, simplyhentai

1.8.7 - 2019-06-28

Additions

Changes

  • Delete empty archives created by the zip post-processor (#316)

Fixes

  • Handle hitomi Game CG galleries correctly (#321)

  • Miscellaneous fixes for deviantart, hitomi, pururin, kissmanga, keenspot, mangoxo, imagefap

1.8.6 - 2019-06-14

Additions

Changes

  • Detect directlink URLs with upper case filename extensions (#296)

Fixes

  • Improved error handling for tumblr API calls (#297)

  • Fixed extraction of livedoor blogs (#301)

  • Fixed extraction of special deviantart Sta.sh items (#307)

  • Fixed pagination for specific keenspot comics

1.8.5 - 2019-06-01

Additions

Changes

  • Adjust wallhaven extractors to new page layout:

    • use API and add api-key option

    • removed traditional login support

  • Provide original filenames for patreon downloads (#268)

  • Use e-hentai.org or exhentai.org depending on input URL (#278)

Fixes

  • Fix pagination over sankaku popular listings (#265)

  • Fix folder and collection extraction on deviantart (#271)

  • Detect "AreYouHuman" redirects on readcomiconline (#279)

  • Miscellaneous fixes for hentainexus, livedoor, ngomik

1.8.4 - 2019-05-17

Additions

Changes

  • Improved flickr metadata and video extraction (#246)

Fixes

  • Download original GIF animations from deviantart (#242)

  • Ignore missing edge_media_to_comment fields on instagram (#250)

  • Fix serialization of datetime objects for --write-metadata (#251, #252)

  • Allow multiple post-processor command-line options at once (#253)

  • Prevent crash on booru sites when no tags are available (#259)

  • Fix extraction on instagram after rhx_gis field removal (#266)

  • Avoid Cloudflare CAPTCHAs for Python interpreters built against OpenSSL < 1.1.1

  • Miscellaneous fixes for luscious

1.8.3 - 2019-05-04

Additions

Changes

  • Standalone executables are now built using PyInstaller:

    • uses the latest CPython interpreter (Python 3.7.3)

    • available on several platforms (Windows, Linux, macOS)

    • includes the certifi CA bundle, youtube-dl, and pyOpenSSL on Windows

Fixes

  • Patch urllib3's default list of SSL/TLS ciphers to prevent Cloudflare CAPTCHAs (#227) (Windows users need to install pyOpenSSL for this to take effect)

  • Provide fallback URLs for twitter images (#237)

  • Send Referer headers when downloading from hitomi (#239)

  • Updated login procedure on mangoxo

1.8.2 - 2019-04-12

Additions

Changes

  • Increased required Requests version to 2.11.0

Fixes

  • Improved image quality on reactor sites (#210)

  • Support imagebam galleries with more than 100 images (#219)

  • Updated Cloudflare bypass code

1.8.1 - 2019-03-29

Additions

Changes

  • Better FFmpeg arguments for --ugoira-conv

  • Adjusted metadata for luscious albums

Fixes

  • Proper handling of instagram multi-image posts (#178, #201)

  • Fixed tumblr avatar URLs when not using OAuth1.0 (#193)

  • Miscellaneous fixes for exhentai, komikcast

1.8.0 - 2019-03-15

Additions

Changes

  • More metadata for nhentai galleries

  • Combined myportfolio extractors into one

  • Renamed name metadata field to filename and removed the original filename field

  • Simplified and improved internal data structures

  • Optimized creation of child extractors

Fixes

  • Filter empty tumblr URLs (#165)

  • Filter ads and improve connection speed on hentaifoundry

  • Show proper error messages if luscious galleries are unavailable

  • Miscellaneous fixes for mangahere, ngomik, simplyhentai, imgspice

Removals

  • seaotterscans

1.7.0 - 2019-02-05

1.6.3 - 2019-01-18

  • Added metadata post-processor to write image metadata to an external file (#135)

  • Added option to reverse chapter order of manga extractors (#149)

  • Added authentication support for danbooru (#151)

  • Added tag metadata for exhentai and hbrowse galleries

  • Improved *reactor extractors (#148)

  • Fixed extraction issues for nhentai (#156), pinterest, mangapark

1.6.2 - 2019-01-01

  • Added support for:

  • Added support for multiple items on sta.sh pages (#113)

  • Added option to download tumblr avatars (#137)

  • Changed defaults for visited post types and inline media on tumblr

  • Improved inline extraction of tumblr posts (#133, #137)

  • Improved error handling and retry behavior of all API calls

  • Improved handling of missing fields in format strings (#136)

  • Fixed hash extraction for unusual tumblr URLs (#129)

  • Fixed image subdomains for hitomi galleries (#142)

  • Fixed and improved miscellaneous issues for kissmanga (#20), luscious, mangapark, readcomiconline

1.6.1 - 2018-11-28

1.6.0 - 2018-11-17

  • Added support for:

  • Added youtube-dl integration and video downloads for twitter (#99), behance, artstation

  • Added per-extractor options for network connections (retries, timeout, verify)

  • Added a --no-check-certificate command-line option

  • Added ability to specify the number of skipped downloads before aborting/exiting (#115)

  • Added extractors for scraps, favorites, popular and recent images on hentaifoundry (#110)

  • Improved login procedure for pixiv to avoid unwanted emails on each new login

  • Improved album metadata and error handling for flickr (#109)

  • Updated default User-Agent string to Firefox 62 (#122)

  • Fixed twitter API response handling when logged in (#123)

  • Fixed issue when converting Ugoira using H.264

  • Fixed miscellaneous issues for 2chan, deviantart, fallenangels, flickr, imagefap, pinterest, turboimagehost, warosu, yuki (#112)

1.5.3 - 2018-09-14

  • Added support for:

  • Added black-/whitelist options for post-processor modules

  • Added support for tumblr inline videos (#102)

  • Fixed extraction of smugmug albums without owner (#100)

  • Fixed issues when using default config values with reddit extractors (#104)

  • Fixed pagination for user favorites on sankaku (#106)

  • Fixed a crash when processing deviantart journals (#108)

1.5.2 - 2018-08-31

  • Added support for twitter timelines (#96)

  • Added option to suppress FFmpeg output during ugoira conversions

  • Improved filename formatter performance

  • Improved inline image quality on tumblr (#98)

  • Fixed image URLs for newly released mangadex chapters

  • Fixed a smaller issue with deviantart journals

  • Replaced subapics with ngomik

1.5.1 - 2018-08-17

  • Added support for:

  • Added support for related pins on pinterest

  • Fixed accessing "offensive" galleries on exhentai (#97)

  • Fixed extraction issues for mangadex, komikcast and behance

  • Removed original-image functionality from tumblr, since "raw" images are no longer accessible

1.5.0 - 2018-08-03

  • Added support for:

  • Added custom format string options to handle long strings (#92, #94)

    • Slicing: "{field[10:40]}"

    • Replacement: "{field:L40/too long/}"

  • Improved frame rate handling for ugoira conversions

  • Improved private access token usage on deviantart

  • Fixed metadata extraction for some images on nijie

  • Fixed chapter extraction on mangahere

  • Removed whatisthisimnotgoodwithcomputers

  • Removed support for Python 3.3

1.4.2 - 2018-07-06

  • Added image-pool extractors for safebooru and rule34

  • Added option for extended tag information on booru sites (#92)

  • Added support for DeviantArt's new URL format

  • Added support for mangapark mirrors

  • Changed imagefap extractors to use HTTPS

  • Fixed crash when skipping downloads for files without known extension

1.4.1 - 2018-06-22

  • Added an ugoira post-processor to convert pixiv animations to WebM

  • Added --zip and --ugoira-conv command-line options

  • Changed how ugoira frame information is handled

    • instead of being written to a separate file, it is now made available as metadata field of the ZIP archive

  • Fixed manga and chapter titles for mangadex

  • Fixed file deletion by post-processors

1.4.0 - 2018-06-08

  • Added support for:

  • Added extractors for

    • pixiv search results and followed users

    • deviantart search results and popular listings

  • Added post-processors to perform actions on downloaded files

  • Added options to configure logging behavior

  • Added OAuth support for smugmug

  • Changed pixiv extractors to use the AppAPI

    • this breaks favorite archive IDs and changes some metadata fields

  • Changed the default filename format for tumblr and renamed offset to num

  • Fixed a possible UnicodeDecodeError during installation (#86)

  • Fixed extraction of mangadex manga with more than 100 chapters (#84)

  • Fixed miscellaneous issues for imgur, reddit, komikcast, mangafox and imagebam

1.3.5 - 2018-05-04

  • Added support for:

  • Added title information for mangadex chapters

  • Improved the pinterest API implementation (#83)

  • Improved error handling for deviantart and tumblr

  • Removed gomanga and puremashiro

1.3.4 - 2018-04-20

  • Added support for custom OAuth2 credentials for pinterest

  • Improved rate limit handling for tumblr extractors

  • Improved hentaifoundry extractors

  • Improved imgur URL patterns

  • Fixed miscellaneous extraction issues for luscious and komikcast

  • Removed loveisover and spectrumnexus

1.3.3 - 2018-04-06

  • Added extractors for

    • nhentai search results

    • exhentai search results and favorites

    • nijie doujins and favorites

  • Improved metadata extraction for exhentai and nijie

  • Improved tumblr extractors by avoiding unnecessary API calls

  • Fixed Cloudflare DDoS protection bypass

  • Fixed errors when trying to print unencodable characters

1.3.2 - 2018-03-23

  • Added extractors for artstation albums, challenges and search results

  • Improved URL and metadata extraction for hitomiand nhentai

  • Fixed page transitions for danbooru API results (#82)

1.3.1 - 2018-03-16

  • Added support for:

  • Added Cloudflare DDoS protection bypass to komikcast extractors

  • Changed archive ID formats for deviantart folders and collections

  • Improved error handling for deviantart API calls

  • Removed imgchili and various smaller image hosts

1.3.0 - 2018-03-02

  • Added --proxy to explicitly specify a proxy server (#76)

  • Added options to customize archive ID formats and undefined replacement fields

  • Changed various archive ID formats to improve their behavior for favorites / bookmarks / etc.

    • Affected modules are deviantart, flickr, tumblr, pixiv and all …boorus

  • Improved sankaku and idolcomplex support by

    • respecting page and next URL parameters (#79)

    • bypassing the page-limit for unauthenticated users

  • Improved directlink metadata by properly unquoting it

  • Fixed pixiv ugoira extraction (#78)

  • Fixed miscellaneous extraction issues for mangastream and tumblr

  • Removed yeet, chronos, coreimg, hosturimage, imageontime, img4ever, imgmaid, imgupload

1.2.0 - 2018-02-16

  • Added support for:

  • Added --download-archive to record downloaded files in an archive file

  • Added --write-log to write logging output to a file

  • Added a filetype check on download completion to fix incorrectly assigned filename extensions (#63)

  • Added the tumblr:... pseudo URI scheme to support custom domains for Tumblr blogs (#71)

  • Added fallback URLs for tumblr images (#64)

  • Added support for reddit-hosted images (#68)

  • Improved the input file format by allowing comments and per-URL options

  • Fixed OAuth 1.0 signature generation for Python 3.3 and 3.4 (#75)

  • Fixed smaller issues for luscious, hentai2read, hentaihere and imgur

  • Removed the batoto module

1.1.2 - 2018-01-12

  • Added support for:

  • Added an option to filter reblogs on tumblr (#61)

  • Added OAuth user authentication for tumblr (#65)

  • Added support for slideshare mobile URLs (#67)

  • Improved pagination for various …booru sites to work around page limits

  • Fixed chapter information parsing for certain manga on kissmanga (#58) and batoto (#60)

1.1.1 - 2017-12-22

  • Added support for:

  • Added pool- and post-extractors for sankaku

  • Added OAuth user authentication for deviantart

  • Updated luscious to support members.luscious.net URLs (#55)

  • Updated mangahere to use their new domain name (mangahere.cc) and support mobile URLs

  • Updated gelbooru to not be restricted to the first 20,000 images (#56)

  • Fixed extraction issues for nhentai and khinsider

1.1.0 - 2017-12-08

  • Added the -r/--limit-rate command-line option to set a maximum download rate

  • Added the --sleep command-line option to specify the number of seconds to sleep before each download

  • Updated gelbooru to no longer use their now disabled API

  • Fixed SWF extraction for sankaku (#52)

  • Fixed extraction issues for hentai2read and khinsider

  • Removed the deprecated --images and --chapters options

  • Removed the mangazuki module

1.0.2 - 2017-11-24

  • Added an option to set a custom user-agent string

  • Improved retry behavior for failed HTTP requests

  • Improved seiga by providing better metadata and getting more than the latest 200 images

  • Improved tumblr by adding support for all post types, scanning for inline images and following external links (#48)

  • Fixed extraction issues for hbrowse, khinsider and senmanga

1.0.1 - 2017-11-10

  • Added support for:

  • Fixed exception handling during file downloads which could lead to a premature exit

  • Fixed an issue with tumblr where not all images would be downloaded when using tags (#48)

  • Fixed extraction issues for imgbox (#47), mangastream (#49) and mangahere

1.0.0 - 2017-10-27

  • Added support for:

  • Added support for pixiv ranking lists

  • Added support for booru popular lists (danbooru, e621, konachan, yandere, 3dbooru)

  • Added the --cookies command-line and cookies config option to load additional cookies

  • Added the --filter and --chapter-filter command-line options to select individual images or manga-chapters by their metadata using simple Python expressions (#43)

  • Added the verify config option to control certificate verification during file downloads

  • Added config options to overwrite internally used API credentials (API Tokens & IDs)

  • Added -K as a shortcut for --list-keywords

  • Changed the --images and --chapters command-line options to --range and --chapter-range

  • Changed keyword names for various modules to make them accessible by --filter. In general minus signs have been replaced with underscores (e.g. gallery-id -> gallery_id).

  • Changed default filename formats for manga extractors to optionally use volume and title information

  • Improved the downloader modules to use .part files and support resuming incomplete downloads (#29)

  • Improved deviantart by distinguishing between users and groups (#26), always using HTTPS, and always downloading full-sized original images

  • Improved sankaku by adding authentication support and fixing various other issues (#44)

  • Improved URL pattern for direct image links (#30)

  • Fixed an issue with luscious not getting original image URLs (#33)

  • Fixed various smaller issues for batoto, hentai2read (#38), jaiminisbox, khinsider, kissmanga (#28, #46), mangahere, pawoo, twitter

  • Removed kisscomic and yonkouprod modules

0.9.1 - 2017-07-24

0.9.0 - 2017-06-28

  • Added support for:

  • Added support for direct image links

  • Added user authentication via OAuth for reddit and flickr

  • Added support for user authentication data from .netrc files (#22)

  • Added a simple progress indicator for multiple URLs (#19)

  • Added the --write-unsupported command-line option to write unsupported URLs to a file

  • Added documentation for all available config options (configuration.rst)

  • Improved pixiv to support tags for user downloads (#17)

  • Improved pixiv to support shortened and http://pixiv.me/... URLs (#23)

  • Improved imgur to properly handle .gifv images and provide better metadata

  • Fixed an issue with kissmanga where metadata parsing for some series failed (#20)

  • Fixed an issue with getting filename extensions from Content-Type response headers

0.8.4 - 2017-05-21

  • Added the --abort-on-skip option to stop extraction if a download would be skipped

  • Improved the output format of the --list-keywords option

  • Updated deviantart to support all media types and journals

  • Updated fallenangels to support their Vietnamese version

  • Fixed an issue with multiple tags on ...booru sites

  • Removed the yomanga module

0.8.3 - 2017-05-01

  • Added support for https://pawoo.net/

  • Added manga extractors for all FoOlSlide-based modules

  • Added the -q/--quiet and -v/--verbose options to control output verbosity

  • Added the -j/--dump-json option to dump extractor results in JSON format

  • Added the --ignore-config option

  • Updated the exhentai extractor to fall back to using the e-hentai version if no username is given

  • Updated deviantart to support sta.sh URLs

  • Fixed an issue with kissmanga which prevented image URLs from being decrypted properly (again)

  • Fixed an issue with pixhost where for an image inside an album it would always download the first image of that album (#13)

  • Removed the mangashare and readcomics modules

0.8.2 - 2017-04-10

  • Fixed an issue in kissmanga which prevented image URLs from being decrypted properly

0.8.1 - 2017-04-09

0.8.0 - 2017-03-28

  • Added logging support

  • Added the -R/--retries option to specify how often a download should be retried before giving up

  • Added the --http-timeout option to set a timeout for HTTP connections

  • Improved error handling/tolerance during HTTP file downloads (#10)

  • Improved option parsing and the help message from -h/--help

  • Changed the way configuration values are used by prioritizing top-level values

    • This allows for cmdline options like -u/--username to overwrite values set in configuration files

  • Fixed an issue with imagefap.com where incorrectly reported gallery sizes would cause the extractor to fail (#9)

  • Fixed an issue with seiga.nicovideo.jp where invalid characters in an API response caused the XML parser to fail

  • Fixed an issue with seiga.nicovideo.jp where the filename extension for the first image would be used for all others

  • Removed support for old configuration paths on Windows

  • Removed several modules:

    • mangamint: site is down

    • whentai: now requires account with VIP status for original images

    • kissmanga: encrypted image URLs (will be re-added later)

0.7.0 - 2017-03-06

  • Added --images and --chapters options

    • Specifies which images (or chapters) to download through a comma-separated list of indices or index-ranges

    • Example: --images -2,4,6-8,10- will select images with index 1, 2, 4, 6, 7, 8 and 10 up to the last one

  • Changed the -g/--get-urls option

    • The amount of how often the -g option is given now determines up until which level URLs are resolved.

    • See 3bca86618505c21628cd9c7179ce933a78d00ca2

  • Changed several option keys:

    • directory_fmt -> directory

    • filename_fmt -> filename

    • download-original -> original

  • Improved FoOlSlide-based extractors

  • Fixed URL extraction for hentai2read

  • Fixed an issue with deviantart, where the API access token wouldn't get refreshed

0.6.4 - 2017-02-13

  • Added new extractors:

  • Fixed url- and data-extraction for:

    • nhentai

    • mangamint

    • twitter

    • imagetwist

  • Disabled InsecureConnectionWarning when no certificates are available

0.6.3 - 2017-01-25

  • Added new extractors:

    • gomanga

    • yomanga

    • mangafox

  • Fixed deviantart extractor failing - switched to using their API

  • Fixed an issue with SQLite on Python 3.6

  • Automated test builds via Travis CI

  • Standalone executables for Windows

0.6.2 - 2017-01-05

  • Added new extractors:

    • kisscomic

    • readcomics

    • yonkouprod

    • jaiminisbox

  • Added manga extractor to batoto-module

  • Added user extractor to seiga-module

  • Added -i/--input-file argument to allow local files and stdin as input (like wget)

  • Added basic support for file:// URLs

    • this allows for the recursive extractor to be applied to local files:

    • $ gallery-dl r:file://[path to file]

  • Added a utility extractor to run unit test URLs

  • Updated luscious to deal with API changes

  • Fixed twitter to provide the original image URL

  • Minor fixes to hentaifoundry

  • Removed imgclick extractor

0.6.1 - 2016-11-30

  • Added new extractors:

    • whentai

    • readcomiconline

    • sensescans, worldthree

    • imgmaid, imagevenue, img4ever, imgspot, imgtrial, pixhost

  • Added base class for extractors of FoOlSlide-based sites

  • Changed default paths for configuration files on Windows

    • old paths are still supported, but that will change in future versions

  • Fixed aborting downloads if a single one failed (#5)

  • Fixed cloudflare-bypass cache containing outdated cookies

  • Fixed image URLs for hitomi and 8chan

  • Updated deviantart to always provide the highest quality image

  • Updated README.rst

  • Removed doujinmode extractor

0.6.0 - 2016-10-08

  • Added new extractors:

    • hentaihere

    • dokireader

    • twitter

    • rapidimg, picmaniac

  • Added support to find filename extensions by Content-Type response header

  • Fixed filename/path issues on Windows (#4):

    • Enable path names with more than 260 characters

    • Remove trailing spaces in path segments

  • Updated Job class to automatically set category/subcategory keywords

0.5.2 - 2016-09-23

  • Added new extractors:

    • pinterest

    • rule34

    • dynastyscans

    • imagebam, coreimg, imgcandy, imgtrex

  • Added login capabilities for batoto

  • Added --version cmdline argument to print the current program version and exit

  • Added --list-extractors cmdline argument to print names of all extractor classes together with descriptions and example URLs

  • Added proper error messages if an image/user does not exist

  • Added unittests for every extractor

0.5.1 - 2016-08-22

  • Added new extractors:

    • luscious

    • doujinmode

    • hentaibox

    • seiga

    • imagefap

  • Changed error output to use stderr instead of stdout

  • Fixed broken pipes causing an exception-dump by catching BrokenPipeErrors

0.5.0 - 2016-07-25

0.4.1 - 2015-12-03

  • New modules (imagetwist, turboimagehost)

  • Manga-extractors: Download entire manga and not just single chapters

  • Generic extractor (provisional)

  • Better and configurable console output

  • Windows support

0.4.0 - 2015-11-26

0.3.3 - 2015-11-10

0.3.2 - 2015-11-04

0.3.1 - 2015-10-30

0.3.0 - 2015-10-05

0.2.0 - 2015-06-28

0.1.0 - 2015-05-27