Mechanical Edits/Mateusz Konieczny - bot account/remove tracking parameters2/

From OpenStreetMap Wiki
Jump to navigation Jump to search

Page content created as advised on Automated_Edits_code_of_conduct#Document_and_discuss_your_plans.

Who

I, Mateusz Konieczny using my bot account

contact

message via OSM I will respond also to PMs to the bot account. In both cases I will be notified about incoming PMs via email and notifications in OSM editors.

What

URL often have unnecessary parts, typically added for tracking purposes. This tracking parameters should never appear in any osm tags.

FB, Google and other add tracking links for various purposes.

It means that it is beneficial to turn tag

website=http://paris.intersquat.org/les-lieux/le-satellite/?fbclid=de58e340d6aa79a584552a2055042d004b9b19454bc0d7a6046fc81fc90f51

into

website=http://paris.intersquat.org/les-lieux/le-satellite/

Usually tracking links are added by clueless people who just searched for a website and copied it from FB/Google.

There are rare cases of links created to specifically track OSM users, see for example

In general I have not noticed correlation between presence of tracking links and additional issues that would not be detected automatically.

Therefore automatic removal of tracking parameters is not causing loss of useful indicators of areas that should be reviewed.

Osmose and JOSM validators and StreetComplete are offering better indicators.

(If anyone is interested in list of more systematic issues that are automatically detectable but require human to fix - please contact me, I have found more broken imports, data with suspicious copyright status, bad tagging than I can process).

Automatic removal would allow me to spend time on something more useful, than reviewing all cases where this links are present and confirming them one by one.

Proposed bot edit would remove links where all used parameters are tracking users and may be removed. Other links will be reviewed manually to catch also currently unknown tracking parameters.

Anchors (#section) will be preserved. Code is tested, was using it in a manual review mode and for a fully automated edit run that removed tracking parameters from over 1000 objects - see https://wiki.openstreetmap.org/wiki/Mechanical_Edits/Mateusz_Konieczny_-_bot_account/remove_tracking_parameters

I have experience with automated edits, see https://wiki.openstreetmap.org/wiki/Mechanical_Edits/Mateusz_Konieczny_-_bot_account

Yes, editing element will cause it to be edited and change "last edited" date. Effect will be exactly the same in case of using bot and manual edit (which I will do anyway in case of rejecting this automated edit proposal). Note that in case of bot edits you may filter out bot edits marked as automatic.

Why

Tracking parameters is not welcomed and is explicitly discouraged in links added as values into OSM database. For start, such parameter add nothing useful and make link more complex. Additionally, such tracking is unwanted, undesirable and unacceptable.

Numbers

About 1300 objects. changes listing: https://gist.github.com/matkoniecz/1d20caa198ec2d4001d95adf09123a8a (based on current OSM database, if OSM data changes then actual edit will be different, feel free to make backup of this linked file - it may be deleted some time after edit) (impossible to publish on OSM Wiki due to triggering spam filter)

How

Changesets will be split in parts to avoid covering huge areas or massive number of objects. In case of object itself being extremely large, larger than desired bounding box some oversized changeset areas are unavoidable (for example, in case of editing country boundary).

Bot will sleep between changesets to reduce risks of unexpected behavior and give more time to react if things are not going well and to eliminate risks of affecting OSM performance by making many edits at the same time.

  • Bot will edit link to remove undesirable parts (tracking parameters)
  • link in any tag value will be checked
    • Edit will not be done of url has no parameters
    • Edit will be done of url has parameters and all of them are tracking ones or parameters explicitly whitelisted
      • Tracking parameters will be removed

state before a mechanical edit - example based on https://www.openstreetmap.org/node/4636662880 :

state after a mechanical edit:


Discussion

https://lists.openstreetmap.org/pipermail/talk/2021-February/086076.html

Bot source code

Bot is using https://github.com/matkoniecz/osm_bot_abstraction_layer library, this code is GNU GPLv3 licensed

from osm_bot_abstraction_layer.generic_bot_retagging import run_simple_retagging_task
import re
import time
import datetime

def main():
    run_in_bot_mode_early_2021()

def run_in_manual_mode():
    test_expectations()
    print(datetime.datetime.now())
    print(query_of_affected_items())
    run_simple_retagging_task(
        max_count_of_elements_in_one_changeset=500,
        objects_to_consider_query=query_of_affected_items(),
        objects_to_consider_query_storage_file='/media/mateusz/OSM_cache/OSM-cache/overpass/osm_elements_with_trackers.osm',
        is_in_manual_mode=True,
        changeset_comment='remove tracking parameters',
        discussion_url='not necessary, as edit was manually reviewed and tracker parameters are clearly unwanted',
        osm_wiki_documentation_page='not necessary, as edit was manually reviewed',
        edit_element_function=edit_element_manual,
    )
    print(datetime.datetime.now())

def run_in_bot_mode_early_2021():
    """
    it will list of planned edits
    and ask for stop/go before doing any edits
    this can be used to generate list of simulated edits before getting bot approval
    """
    test_expectations()
    print(datetime.datetime.now())
    print(query_of_affected_items())
    run_simple_retagging_task(
        max_count_of_elements_in_one_changeset=500,
        objects_to_consider_query=query_of_affected_items(),
        objects_to_consider_query_storage_file='/media/mateusz/OSM_cache/OSM-cache/overpass/osm_elements_with_trackers.osm',
        is_in_manual_mode=False,
        changeset_comment='remove tracking parameters from links in OSM tags',
        discussion_url='https://lists.openstreetmap.org/pipermail/talk/2021-February/thread.html#86076',
        osm_wiki_documentation_page='https://wiki.openstreetmap.org/wiki/Mechanical_Edits/Mateusz_Konieczny_-_bot_account/remove_tracking_parameters2/',
        edit_element_function=edit_element_automatic,
    )
    print(datetime.datetime.now())

"""
tl;dr:

I propose to run an automated bot edit that will remove tracking parameters, turning tags such as
website=http://paris.intersquat.org/les-lieux/le-satellite/?fbclid=de58e340d6aa79a584552a2055042d004b9b19454bc0d7a6046fc81fc90f51
into
website=http://paris.intersquat.org/les-lieux/le-satellite/


I did it already before, see: https://wiki.openstreetmap.org/wiki/Mechanical_Edits/Mateusz_Konieczny_-_bot_account/remove_tracking_parameters
This edit will remove newly edited links and purge more tracking parameters.

If anything will go wrong I will fix it.
I have experience with automated edits, see
https://wiki.openstreetmap.org/wiki/Mechanical_Edits/Mateusz_Konieczny_-_bot_account

changes listing: https://gist.github.com/matkoniecz/1d20caa198ec2d4001d95adf09123a8a
(based on current OSM database, if OSM data changes then actual edit will be different,
feel free to make backup of this linked file - it may be deleted some time after edit)

source code and other documentation (except source code, duplicate of that posting):
https://wiki.openstreetmap.org/wiki/Mechanical_Edits/Mateusz_Konieczny_-_bot_account/remove_tracking_parameters2/

--------------------------------------


I propose to run a scripted edit - it was already run before but this will
remove more tracking parameters.

URL often have unnecessary parts, some added for tracking purposes 
by FB, Google and others.
This tracking parameters should never appear in any osm tags.

It means that it is beneficial to turn tag
website=http://paris.intersquat.org/les-lieux/le-satellite/?fbclid=de58e340d6aa79a584552a2055042d004b9b19454bc0d7a6046fc81fc90f51
into
website=http://paris.intersquat.org/les-lieux/le-satellite/
and it is worth doing it as an edit.

This urls can be often fixed using an automated script, allowing to
use human time on something more productive.

Human-made edit will also result in changing "last edited by"
(while not allowing to filter out such edits unlike marked bot edit),
there are better ways to spot areas requiring fixes and we are not lacking
places with QA indicators that manual review is needed.

Usually tracking links are added by clueless people who just searched for 
a website and copied it from FB/Google.

There are rare cases of links created to specifically track OSM users
see for example
* https://www.openstreetmap.org/way/754704241/history
** https://www.cronauerlaw.com/?utm_source=openstreetmap
* https://www.openstreetmap.org/node/1063808111/history
** http://www.travelerscoffee.ru?utm_campaign=geo&utm_source=openstreetmap&utm_medium=link
* https://www.openstreetmap.org/node/6817678019/history
** https://www.resotainer.fr/agence-bonneuil-sur-marne?utm_source=open-street-map&utm_medium=recherche-locale&utm_content=openstreetmap&utm_campaign=open-street-map-garde-meubles-bonneuil-sur-marne
* https://www.openstreetmap.org/node/1684317522
** http://www.travelerscoffee.ru?utm_campaign=geo&utm_source=openstreetmap&utm_medium=link

In general I have not noticed correlation between presence of tracking links
and additional issues that would not be detected automatically.

Therefore automatic removal of tracking parameters is not causing loss of 
useful indicators of areas that should be reviewed.
Osmose and JOSM validators and StreetComplete are offering many better indicators,
and we are not in danger of running out of places where human intervention is clearly needed.

Automatic removal would allow me and others to spend time on something more useful,
than reviewing all cases where tracking is clearly present and confirming them one by one.

Proposed bot edit would remove links where all used parameters are tracking
users and may be removed. 

I am reviewing manually more complicated cases to catch
also currently unknown tracking parameters.

Anchors (#section) will be preserved.

Code is tested, was using it in a manual review mode and for a fully automated edit run
that removed tracking parameters from over 1000 objects - see
https://wiki.openstreetmap.org/wiki/Mechanical_Edits/Mateusz_Konieczny_-_bot_account/remove_tracking_parameters

I have experience with automated edits, see
https://wiki.openstreetmap.org/wiki/Mechanical_Edits/Mateusz_Konieczny_-_bot_account

Yes, editing element will cause it to be edited and change "last edited" date.
Effect will be exactly the same in case of using bot and manual edit
(which I will do anyway in case of rejecting this automated edit proposal).
Note that in case of bot edits you may filter out bot edits marked as automatic.
"""

def malicious_parameters_for_eradication_removed_and_triggering_inclusion():
    # gclsrc https://developers.google.com/search-ads/v2/how-tos/conversions/
    # dclid https://support.google.com/searchads/answer/6292795?hl=en 
    # wt.tsrc  https://docs.oracle.com/en/cloud/saas/marketing/infinity-user/Help/parameters/data_augmentation/traffic_source_and_search.htm 
    # utm_id https://matomo.org/faq/general/faq_119/
    # yclid https://github.com/Smile4ever/Neat-URL
    # trkCampaign https://github.com/Smile4ever/Neat-URL
    # mkt_tok https://github.com/Smile4ever/Neat-URL
    # sc_campaign https://github.com/Smile4ever/Neat-URL
    # sc_channel https://github.com/Smile4ever/Neat-URL
    # sc_content https://github.com/Smile4ever/Neat-URL
    # sc_medium https://github.com/Smile4ever/Neat-URL
    # sc_outcome https://github.com/Smile4ever/Neat-URL
    # sc_geo https://github.com/Smile4ever/Neat-URL
    # sc_country https://github.com/Smile4ever/Neat-URL
    # igshid - looks like instagram tracking link (not just me - see https://www.bradymoritz.com/igshid-the-new-instagram-click-tracking-id/ )
    # "mbid", "cmpid", "c_id", "campaign_id", "Campaign", "fb_action_ids", "fb_action_types", "fb_ref", "fb_source", "gs_l", _hsenc https://github.com/Smile4ever/Neat-URL
    # utm_campain - variant of utm_campaign seen in wild
    return [
            "fbclid", "gclid", "campaign_ref", "mc_id",
            "utm_source", "utm_medium", "utm_term", "utm_content", "utm_campaign", "utm_id",
            "gclsrc", "dclid", "wt.tsrc", "WT.tsrc", "zanpid", "yclid", "utm_campain", 
            "trkCampaign", "mkt_tok", "sc_campaign", "sc_channel", "sc_content",
            "sc_medium", "sc_outcome", "sc_geo", "sc_country", "mbid", "cmpid",
            "campaign_id", "Campaign", "fb_action_ids",
            "fb_action_types", "fb_ref", "fb_source", "gs_l", "_hsenc",
            "igshid", "CampIDMin", "CampIDMaj", "campaign", "Campaign", "campaignid", "campaignId", "adid",
            "adgroupid", "refr", "referrer", "cm_mmc", "lw_cmp", "CLID",
            "ReferralSource", "SourceID", "trkid", "adjust_creative", "partner_slug", "y_source",
            "oppartnerid", "padid", "otppartnerid", "ref_device_id", "utm_kxconfid", "SEO_id",
            "originalReferrer", "spMailingID", "hsCtaTracking",
            ]

def malicious_parameters_removed_in_manual_mode():
            # disable in automation as not 100% sure
    return ["ei", "c_id", "s_cid", "channel", "y_source", "channel", "source", "source_impression_id", "adh_i", "facilitatorId",
            "cid", #sometimes used for tracking, sometimes a real parameter
            "lpc", # rare, tracking in spotted cases
            'sc_cmp' #????
            ]
    # ei https://stackoverflow.com/questions/18584386/what-does-ei-mean-in-the-google-homepage-url-https-www-google-co-in-gws-rd
    
    # ????????
    #"_hsmi", "__hssc", "__hstc", "hsCtaTracking"]},
    # more 
    # https://github.com/Smile4ever/Neat-URL
    # https://github.com/wistia/fresh-url/blob/master/lib/fresh_url.coffee#L92

def parameters_assumed_to_be_ok():
    # yes, websites may start using them instead of more obvious trackers but it have not happened so far
    return [
        "storecode", "storeCode", "shopid", "storeId", "StoreName", "propertycode", "fileId", "stationid", "country_id", "start_id",
        "store-id", "store_code", "objectid", "StoreID", "storeID", "store",
        "locationTypeQ", "locationType", "locationID", "radius", # https://tools.usps.com/go/POLocatorDetailsAction!input.action?&radius=20&locationType=po&locationTypeQ=po&locationID=1474099
        "phone", # https://api.whatsapp.com/send?phone=393913134470
        "address", "location",
        "provinceID", "storeNo", "store_name", "RestaurantID", "iata",
        "latitude", "longitude", "bbox", "lang", "langId", "grop", "loc"
        "magasin", # https://www.intersport-rent.fr/rent/page/pack.aspx?magasin=994
        "i", "id", #????????????
        "cur", # currency
    ]

def parameters_group(parameters):
    return "(" + "|".join(parameters) + ")"
 
def evil_parameters_group_subset_in_scan_query():
    return "(" + "|".join(malicious_parameters_for_eradication_removed_and_triggering_inclusion()) + ")"

def remove_parameters(link, parameters_group, parameters_specified_as_regexp):
    old_link = None
    while old_link != link:
        old_link = link
        link = remove_parameters_if_sole_parameter(link, parameters_group)
        link = remove_parameters_if_leading_parameter(link, parameters_group)
        link = remove_parameters_if_last_or_in_middle(link, parameters_group)
        link = remove_parameters_if_sole_parameter_with_anchor_at_the_end(link, parameters_group)
        if parameters_specified_as_regexp != None:
            link = re.sub("\?" + parameters_specified_as_regexp + "$", "", link) # sole parameter
            if link.find("panoramio.com"):
                link = re.sub("\?(source=wapi)$", "", link)
                link = re.sub("\?(source=wapi#)$", "", link)
            if link.find("bizjournals.com"):
                # https://www.bizjournals.com/milwaukee/news/2017/04/27/associated-bank-plans-branch-at-two-fifty-office.html?ana=RSS%26s%3Darticle_search
                link = re.sub("\?(ana=[^&#]*article_search)$", "", link)
            if link.find("schuh-germann.de"):
                link = re.sub("\?(sPartner=[^&#]*)$", "", link)
    return link

def remove_parameters_if_last_or_in_middle(link, removed_parameter_group):
    return re.sub("&" + removed_parameter_group + "=[^&#]*", "", link)

def remove_parameters_if_sole_parameter(link, removed_parameter_group):
    return re.sub("\?" + removed_parameter_group + "=[^&#]*$", "", link)

def remove_parameters_if_sole_parameter_with_anchor_at_the_end(link, removed_parameter_group):
    return re.sub("\?" + removed_parameter_group + "=[^&#]*#", "#", link)

def remove_parameters_if_leading_parameter(link, removed_parameter_group):
    return re.sub("\?" + removed_parameter_group + "=[^&#]*&", "?", link)

def regexp_for_special_removal():
    return "(medium=referral|adjust_creative=duckduckgo|source=(adwords|adwords_mis1)|utm|ref=nl_google_brand|page=show_ad|veh=seo)"

def remove_clearly_malicious_parameters(link):
    for_removal = malicious_parameters_for_eradication_removed_and_triggering_inclusion()
    evil_parameters_group = parameters_group(for_removal)
    cleaned_link = remove_parameters(link, evil_parameters_group, regexp_for_special_removal())
    return cleaned_link

def edit_element_automatic(tag_dictionary):
    old_tags = dict(tag_dictionary)
    for key in tag_dictionary.keys():
        if tag_dictionary[key].find("http") == 0 and tag_dictionary[key].find(";") == -1:
 
            link = tag_dictionary[key]
            cleaned_link = remove_clearly_malicious_parameters(link)
 
            if tag_dictionary[key] != cleaned_link:
                if cleaned_link.find("?") != -1:
                    cleaned_link_without_good = remove_parameters(cleaned_link, parameters_group(parameters_assumed_to_be_ok()), None)
                    cleaned_link_without_good_and_manual = remove_parameters(cleaned_link_without_good, parameters_group(malicious_parameters_removed_in_manual_mode()), None)
                    if cleaned_link_without_good_and_manual.find("?") != -1:
                        print("not editing as there are still parameters (possibly trackers), left for a manual review: ", key, "=", tag_dictionary[key])
                        print("cleaned link (remaining parameters are what survived, may be empty if parameters removed in manual were present):", cleaned_link_without_good_and_manual)
                        print()
                        return old_tags # other tags also may be tracking or for removal, review manually
            tag_dictionary[key] = cleaned_link
    return tag_dictionary

def edit_element_manual(tag_dictionary):
    old_tags = dict(tag_dictionary)
    for key in tag_dictionary.keys():
        if tag_dictionary[key].find("http") == 0 and tag_dictionary[key].find(";") == -1:

            link = tag_dictionary[key]
            for_removal = malicious_parameters_for_eradication_removed_and_triggering_inclusion() + malicious_parameters_removed_in_manual_mode()
            for_removal_in_automatic = malicious_parameters_for_eradication_removed_and_triggering_inclusion()
            evil_parameters_group = parameters_group(for_removal)
            evil_parameters_group_in_automatic = parameters_group(for_removal_in_automatic)
            cleaned_link = remove_parameters(link, evil_parameters_group, regexp_for_special_removal())
            cleaned_link_automatic = remove_parameters(link, evil_parameters_group_in_automatic, regexp_for_special_removal())

            if tag_dictionary[key] != cleaned_link:
                if (cleaned_link.find("?") == -1 or cleaned_link.find("?") == (len(cleaned_link) - 1) and (cleaned_link == cleaned_link_automatic)):
                    print("not editing as all parameters were removed, automatic mode would succeed, left for an automatic edit: ", key, "=", tag_dictionary[key])
                    return old_tags # leave it for an automatic processing
            tag_dictionary[key] = cleaned_link
    return tag_dictionary


def query_for_limited_keys():
    return """
[out:xml][timeout:25000];
(
  nwr["website"~""" + '".*http.*\?' + evil_parameters_group_subset_in_scan_query() + '"' + """];
  nwr["url"~""" + '".*http.*\?' + evil_parameters_group_subset_in_scan_query() + '"' + """];
  nwr["source"~""" + '".*http.*\?' + evil_parameters_group_subset_in_scan_query() + '"' + """];
);
out body;
>;
out skel qt;
"""


def query_for_all_keys_but_slow_one_area():
    return """[out:xml][timeout:25000];
    area["name:pl"="Polska"]->.searchArea;
    (
        nwr[~".*"~""" + '".*http.*\?' + evil_parameters_group_subset_in_scan_query() + '"' + """](area.searchArea);
    );
    out body;
    >;
    out skel qt;
    """

def query_for_all_keys_but_slow():
    return """[out:xml][timeout:25000];
    (
        nwr[~".*"~""" + '".*http.*\?' + evil_parameters_group_subset_in_scan_query() + '"' + """];
    );
    out body;
    >;
    out skel qt;
    """

def query_of_affected_items():
    return query_for_all_keys_but_slow()
    
def test_expectations():
    expected = [
        {
        "input": "https://www.example.com/?utm_medium=referrall#anchor",
        "output": "https://www.example.com/#anchor"
        },
        {
        "input": "https://www.example.com?utm_medium=referrall#anchor",
        "output": "https://www.example.com#anchor"
        },
        {
        "input": "https://www.example.com/?utm_medium=referrall",
        "output": "https://www.example.com/"
        },
        {
        "input": "https://www.example.com/?utm_source=evil&utm_medium=referral",
        "output": "https://www.example.com/"
        },
        {
        "input": "https://clubhaus-olympic.business.site/?utm_source=gmb&utm_medium=referral",
        "output": "https://clubhaus-olympic.business.site/"
        },
        {
        "input": "https://www.enrichinghappiness.com/branch/bickford-of-clinton?utm_source=local&utm_medium=yext&utm_campaign=website",
        "output": "https://www.enrichinghappiness.com/branch/bickford-of-clinton"
        },
        {
        "input": "https://www.wanderservice-schwarzwald.de/de/tour/wanderungen/rundwanderung-grillhuette/112160527/?utm_medium=referral&utm_source=embed&utm_campaign=embed-plugin-referral",
        "output": "https://www.wanderservice-schwarzwald.de/de/tour/wanderungen/rundwanderung-grillhuette/112160527/"
        },
        {
        "input": "https://www.greeneking-pubs.co.uk/pubs/greater-london/shepherds-tavern/?utm_source=g_places&utm_medium=locations&utm_campaign=",
        "output": "https://www.greeneking-pubs.co.uk/pubs/greater-london/shepherds-tavern/"
        },
        {
        "input": "https://www.komos-stroy.ru/estates/pokrovskiy?utm_source=Yandex.Direct&utm_medium=cpc&utm_campain=Komos.ZHK-search&utm_content=-307466859&utm_term=новостройка%20на%2010%20лет%20октября%20ижевск#66-3-0",
        "output": "https://www.komos-stroy.ru/estates/pokrovskiy#66-3-0"
        },
        {
        "input": "https://www.hammer-zuhause.de/maerkte/storeDetail?utm_campaign=googlemaps&utm_medium=organic&storeCode=0158&utm_source=uberall&utm_content=25746_Wesseln",
        "output": "https://www.hammer-zuhause.de/maerkte/storeDetail?storeCode=0158"
        },
        {
        "input": "https://instagram.com/cafenimmersatt?utm_source=ig_profile_share&igshid=13usw1rzkwwc2",
        "output": "https://instagram.com/cafenimmersatt"
        },
        {
        "input": "https://example.com?valid_parameter=aaaa&utm_medium=fuck_tracking#anchor",
        "output": "https://example.com?valid_parameter=aaaa#anchor"
        },
        {
        # input is a bit malformed, with the first parameter starting from &
        "input": "https://www.mattressfirm.com/stores/ga/canton/mattress-stores-canton-ga-30114-5513.html?&utm_source=Google",
        "output": "https://www.mattressfirm.com/stores/ga/canton/mattress-stores-canton-ga-30114-5513.html?"
        },
        {
        # input is a bit malformed, with the first parameter starting from &
        "input": "https://www.mattressfirm.com/stores/ga/canton/mattress-stores-canton-ga-30114-5513.html?&utm_source=Google%2520My%2520Business&utm_medium=organic&utm_campaign=GMB&utm_term=Atlanta",
        "output": "https://www.mattressfirm.com/stores/ga/canton/mattress-stores-canton-ga-30114-5513.html?"
        },
    ]
    """
        {
        "input": "",
        "output": ""
        },
    """
    for test in expected:
        cleaned = remove_clearly_malicious_parameters(test["input"])
        if (cleaned != remove_clearly_malicious_parameters(test["output"])):
            print(" received:", cleaned)
            print(" expected:", test["output"])
            print("for input:", test["input"])
            raise "failing to make a proper edit"

    expected = [
        {
        "input": {'whatever': 'http://example.com?fbclid=evil'},
        "output": {'whatever': 'http://example.com'}
        },
        {
        "input": {'whatever': 'http://example.com?fbclid=evil&dududduduuddu=ususu'},
        "output": {'whatever': 'http://example.com?fbclid=evil&dududduduuddu=ususu'}
        },
    ]
    for test in expected:
        cleaned = edit_element_automatic(test["input"])
        if cleaned != test['output']:
            print(" received:", cleaned)
            print(" expected:", test["output"])
            print("for input:", test["input"])
            raise "failing to make a proper edit"

    expected = [
        {
        # blatant skipped for automatic
        "input": {'whatever': 'http://example.com?fbclid=evil'},
        "output": {'whatever': 'http://example.com?fbclid=evil'}
        },
        {
        "input": {'whatever': 'http://example.com?fbclid=evil&dududduduuddu=ususu'},
        "output": {'whatever': 'http://example.com?dududduduuddu=ususu'}
        },
    ]
    for test in expected:
        cleaned = edit_element_manual(test["input"])
        if cleaned != test['output']:
            print(" received:", cleaned)
            print(" expected:", test["output"])
            print("for input:", test["input"])
            raise "failing to make a proper edit"

main()

Repetition

This edit will be done once. Next run will require a separate permission.

Opt-out

Please write at mailing list thread that will appear in Discussion section. Note that in case of opt out the same edit will be done manually, it is impossible to keep tracking parameters in OSM.