Useful tools for maintaining filter lists

Let’s create a curated list of tools that can be useful to filter lists maintainers.

Please suggest other tools in comments.

2 Likes

Until recently, Redundant Rule Checker was my go-to tool to test for Adblock Plus syntaxes. However, it turned out that it didn’t have a 1:1 match with ABP’s actual syntax, effectively meaning that my Nordic list hadn’t actually been supported in ABP for most of 2018… :grimacing:

It’s still good for at least one thing, however, which is to look for rules for different domains with the same rule criteria.

2 Likes

As I use currently…

  • TortoiseGit (Enhanced Git for Windows Shell)
  • Notepad++ (mostly for sorting lists and/or hosts entry, can fix non-CRLF but a bit slower step than TortoiseGitDiff)
  • VSCodium (randomly found from CrazyMax’s PortApps lists and see that’s a FOSS version of VSCode with telemetry-free)
  • uBlock0 and uMatrix’s logger itself
1 Like

A guide on how to host an adblock list online, intended for those who are completely new to making adblock lists.

I myself would be especially interested in AdGuard-syntax compatible version(s) of https://adblockplus.org/en/redundancy_check &/or https://adblockplus.org/en/deregifier . I think https://adblockplus.org/en/abp_linkfactory could be useful to list self-publishers.

3 Likes

If someone is taking the time to make an AdGuard-syntax compatible version, uBO Extended syntax also working would be great :slight_smile:

1 Like

I just now realised that we’ve forgot to mention PyFunceble, which has grown to become a fairly adaptable tool to check whether domains are active or not.

Useful tools from Polish Filters Team:

  1. VICHS (Version Include Checksum Hosts Sort)
    That script can:
  • include sections of filterlists into one file
  • create whitelists from direct blocking rules from local and external filterlist using standard method or badfilter modifier
  • include external filterlists or its sections into the local filterlist
  • combine local and external sections and include it into the local filterlist
  • convert direct blocking rules of local or external filterlists to hosts rules and include it into one file
  • convert regex blocking rules (all or only this with star multipler) of local or external filterlists to Pi-hole regex rules and include it into one file
  • update version and modified date of list
  • add checksum
  • sort lines alphabetically
  • remove empty lines
  • remove whitespace character from the end of the lines
  • add, commit and send filterlist into the git repository (it will ask you about password or you can use machine user with CircleCI and personal access token to send filterlist to git repository)
  • integrate with FOP.py script
    More information about that script are on https://github.com/PolishFiltersTeam/ScriptsPlayground/blob/master/Readme_VICHS.md
  1. FOP (Filter Orderer and Preener) - improved version of well-known script, supports uBO and AdGuard syntax. It sorts domains and filter options in lines and removes duplicates. To run it, you need min. python 3.2, then to start it, just type python3 FOP.py --dir DIR/DIR/DIR_WHERE_SORTING_SHOULD_BEGIN.
  2. GDE (Groups Domains Extractor) - helps in finding all domains of specific group and copies it to clipboard. It also has the ability to save configuration of group to file and load it. It requires python3, gettext and following modules: beautifulsoup4, certifi, urllib3, tldextract, html5lib, pysimplegui and appdirs (you can install all modules with pip3).
3 Likes

Web Sniffer, https://chrome.google.com/webstore/detail/web-sniffer/ndfgffclcpdbgghfgkmooklaendohaef

1 Like