Trolltags are tags that negate/invert meaning of an object. Often used to modify the properties of a well known feature such as highway, building or hotel in a way that radically changes the original meaning of the feature.
It is not OK to use one tag (for example tourism=hotel) and add second tag that negates or massively change its meaning (for example mapping prison using tourism=hotel involuntary=yes instead of using amenity=prison). Additional tags should clarify meaning of main tags rather than negate it.
In general, any tag that must be processed to avoid serious misinterpretation of the feature is a trolltag.
For example somebody wants to produce map of cycleways. Simply processing highway=cycleway, highway=path, highway=footway and standard access tags (especially bicycle=designated) should be enough to avoid listing fake ones. Data consumer in that situation should not be expected to check for proposed=yes, demolished=yes, construction=yes, completely_fictional=yes, operational_status=closed or end_date=1990.
Adding tags like proposed=yes is a really poor idea. It is a trap for data consumers not supporting them and it will lead to invalid and highly misleading data. And data consumers supporting completely broken tagging schemes (like [highway=tertiary; construction=yes] instead of supporting just [highway=construction, construction=tertiary]) encourages usage of this tagging method. The danger is that with more and more data tagged using trolltags other data consumers will either be forced to add support for trolltags or stop using OSM data.
Note that some tags may be OK or trolltag depending on how it is used. For example abandoned=yes. It is perfectly OK to add it to building - after all, abandoned building is still building. But using it on shop=supermarket to indicate that shop is no longer operating and it is impossible to buy anything there (in other words - it is no longer a shop) is not OK and should be tagged in proper way (typically - by deleting shop=supermarket or using a decent Lifecycle prefix such as disused:shop=supermarket).
Another example is combination leisure=swimming_pool & swimming_pool=paddling_pool. Paddling pools is shallow and not suitable for swimming in common sense. It is preferable use leisure=paddling_pool instead.
Note that merely badly named tags are not trolltags: building=unknown_value_should_be_filled is a poor substitute for building=yes but it is not a trolltag, while building=demolished or building=actually_not_building or actually_build_as_church=yes added to building=house would be.