Knowledge:Bots/Requests for approval/ClueBot NG

557:

recognize that certain words may be acceptable if similar words are already used in the article. The naive Bayes classifiers recognize if a certain word appears only in vandalism, or if it sometimes appears in good edits. The naive Bayes classifier also provides for instances of normally bad words to be offset by the presence of other words which are not normally found in vandalism (These lists are NOT predefined - they are empirically determined by analyzing the dataset). Additionally, neither of these factors are used independently - they're fed into a neural network along with many other statistics, so the bot can learn from example in what statistical situations a high Bayesian score, or a word's presence in a certain category, is acceptable. Also, the second Bayes classifier uses sets of two words, instead of one, so phrases like "Pussy cat" would be recognized as primarily belonging to good edits (given a large enough data set). As the bot does not use any sort of heuristics, it cannot be programmed to ignore those certain situations that you list. However, these tags, and others, can be added as inputs to the neural network. It should then learn under what circumstances they contribute to an edit being good or bad. I should also note that, of the many false positives I've examined so far, none of them fit into these categories, so it would seem the neural network already does a good job of determining that the edit is not vandalism in these cases. The entire concept relies on the neural network learning complex patterns and making inferences about the data presented to it, and this requires a large dataset. We currently have a dataset of around 30,000 edits, about 20,000 of which are used for training, and the other 10,000 for testing/trialing. We are working on expanding this. About the whitelisting of edits - Yes, this will be implemented when the bot is actually running live. The reason this is not currently active is that we would like to find as many false positives as possible now, before the bot goes into production, so we can work on fine-tuning the statistical parameters of the neural network. Even if an edit would not be reverted in production due to a whitelist, it's useful to train as an example of what should be considered a good edit. The programmatic structure of the bot is modular, and different mechanisms can easily be added - we plan on only adding whitelisting measures post-neural-net (ie, no heuristics that could cause additional false positives).

3563:

especially if it's the same line that's been edited. If there's a longish gap then it sometimes turns out that it's a case of different editors at an educational establishment editing a page about that establishment and the previous edit was in good faith. That's why I have my anti-vandal tool warn the user rather than revert back further. I have the range set at /16 and (I'm guessing here, I don't have any statistics) I'd say the previous edit is also bad at least 80% of the time. That's nowhere near enough for a bot, of course. BTW, I also have the tool hold off if it would revert to a version by a previously reverted vandal - is that worth considering? My preferred solution would be to hold off if the IPs are in the same /16 range, list the article for attention by a human, and (ultimately) give the bot sysop rights so it can briefly semi-protect the page if it considers that more than one of the IPs in that range has recently made a bad edit to the article.

293:

existing bots have a significant number of false positives - likely higher than 0.5%. If the 0.5% false positive rate is deemed too high, it can be adjusted at any time. This exact number could be put up for discussion before going live. The program is capable of generating graphs comparing the false positive rate with vandalism detection rate - do you think it would be useful to post these and open up a discussion concerning them? I can also post current lists of false positives in the trial dataset - it may be useful to see that most of them are poor quality or borderline edits, and on nearly all of them the user has contributed a very few times (not just as of the time of the edit, but as of the present). In response to your concerns about new users being labeled as vandals, I'm re-running the trial dataset right now, discarding any data about previous user contributions. I will update this with the results.

356:

main namespace articles - it wouldn't apply very well to the talk page. It may be possible to set up a simulated environment with a somewhat less functional version by either holding these inputs constant, or removing them from the neural network. This approach differs from all existing ones that I'm aware of in that this approach combines multiple different methods to catch different classes of vandalism. Most existing approaches are either statistics-based, or language-based. This is both. Another key difference is that existing approaches are designed with the mindset of being a research project, and as such, try to maximize overall accuracy without practical considerations. Cluebot-NG has been designed to be practical from the ground up, in terms of speed, and minimizing false positives even if it means a decrease in overall correctness.

742:

contributions to analyze, and there was no data other than the inserted word, this statistic was the only one that contributed to the score. Because our existing dataset is pretty large, false positives like this (and false positives in general) very rarely occur. These can be fixed in one of two ways. The best way is to increase the size of the dataset. If even a single good edit in the dataset included the word "banana", the neural network would put less stock in it. The other way is to increase the minimum number of dataset appearances of a word before it contributes to the Bayesian score. This is currently set at 4 (barely enough for "banana" to trigger it). We are using our best judgement to adjust these parameters to the optimal values, but as I said, the best way is to increase the dataset size.

4003:

the reverse. But this whole project is very much a volunteer effort and we all have real lives elsewhere, no matter what may seem to be the case here. Secondly, it really would be helpful to have some feedback on our efforts: this is, I understand, a fairly basic result in experimental psychology - performance improves with feedback, even negative feedback, compared with no feedback. Even putting up a message to say "At the current rate we expect to go live with a fully-reviewed dataset in the middle of February" would give us a target to beat. Thirdly, it would be nice to have specific feedback on the quality of the classification of difficult edits. No-one expects an individual thank-you for correctly classifying the addition of a

816:

would add only this word. Bayesian scoring also takes into account other words that are added with it (for example, if "birth" was also added, it appears in 43 good edits and only 15 vandalism edits, so it would bring the score far below threshold). In addition to this, the bot also monitors if words already appear in an article before the edit, so even if the word "penis" alone were added, if it already appeared in the article, the bot may take that into account (however, that may not be enough to have it classified as a good edit - all statistics are taken into account). Also, the bot handles words inside quotes differently, as direct quotes on Knowledge are allowed to contain many things that direct article text should not.

3494:

thing to do is just to keep improving the bot (which I'm doing) and the dataset. It would definitely be possible to post borderline edits somewhere - the neural net generates a score which is compared against a threshold. The threshold (currently around 0.95) is calculated from a given false positive rate at dataset training/trial time. A second threshold could be set somewhat below this, where edits falling into that group could be posted somewhere. At the ~0.95 threshold it's currently catching 60% of vandalism with 0.25% false positives (calculated from the trial dataset). At a threshold of around 0.65, it gets over 90% of vandalism (with about 3% false positives). Maybe a threshold around 0.65 would be useful.

5327:

time than responding to individual false positives - if there isn't time for the latter, there would never be time for the former. Another important consideration is that we cannot improve the dataset by ourselves, and nobody wants to spend time on a review interface for a bot that isn't active. The live edits, even at the current state, are not only extremely worthwhile for Knowledge, but also bring in a steady stream of contributors to continue to help improving the dataset. Stopping the bot now would all but eliminate these contributions, and this would probably mean that it would never actually be approved.

862:

acceptable. They mitigate this to some extent using a user-whitelist, and not reverting edits made by a user with more than a certain number of contributions. But this means that all or nearly all of the false positives are occurring with new users, who probably don't understand why their edit was classified as vandalism, or how to report it (if they even notice). Just by looking at their logic and considering how many legitimate edits could be misclassified by simple heuristics, it can be seen that existing false positive rates should be significantly higher than the values being considered for this bot.

3031:

training. We plan on expanding in the future to handle other namespaces as well, in which case, classifications from other namespaces will be used. We really don't want to assume any bot always makes good edits. Although this is usually the case in practice, we'd prefer to have every edit verified. Just classify these as constructive as usual (unless it's another anti-vandal bot with a false positive or something - in this case, it should probably be skipped). As for the question about being notified of any disagreement, I'll defer that to Cobi (the developer of the interface).

3452:

score at all, it has to appear in a certain minimum number of articles total (currently 6). You bring up a good point about the words in the categories. Right now, it assesses which words belong in context by checking added words against words that already appear on the page. This is usually sufficient, but as you pointed out, does have some holes. I may be able to figure out a way to determine statistical word relations - not as in a Markov chain, or a Bayesian classifier, but in a broader sense to sort-of automatically categorize an article.

1375:(seemingly, redirects and disambiguation pages). We will add these reported false positives to the training dataset, and add special handling code for these, but it won't greatly affect the results of a run with the trial dataset. What I think you want is statistics on exactly how much vandalism is really being caught, versus the false positive rate. For this, we need to have a human go through all the edits that bot has seen, and manually classify them to check for accuracy. We have already designed an interface exactly for this purpose (see 5478:- A couple of users had a problem with the fact that the developers do not personally comment on every false positive. The developers do not have nearly enough time to write a personalized response to each one, but every false positive is submitted to the review interface for verification and dataset use. A confirmation page is being added to the report interface to clarify how the reports are used. A user also suggested periodic overviews of false positive statistics - this may be possible, but difficult, and we are looking into it. 3159:

would end up being 0.5 or less, well below the 0.95 threshold it's currently at. Multiple previous warnings significantly increase the probability that a given edit is classified as vandalism. Increasing dataset size and including instances where users with multiple warnings made constructive edits will decrease this kind of occurrence. Note: Removing this statistic from the neural network decreased catch rate when normalized to the same false positive rate, so this statistic is helpful overall to the performance of the method.

3683:

etc. (I believe these things are being addressed in the code, but I'm not sure about that :D) and it's really good to see these getting resolved. However, the bulk of the false positives seem to be down to not having a large enough dataset. Some of these are understandable, for example edits which in the context of the article are good, but appear to be vandalism otherwise. But the large majority seem to be edits which can't really be said to look anything like vandalism. I'd just like to say I think it's key for this bot to not

4362:

causing the bot's live performance to not measure up to its theoretical performance based on a dataset trial. After rewriting the dataset downloader to use the same code as the live downloader, and regenerating the dataset, the bot's live performance is now much closer to its theoretical dataset performance (before the live bot was catching only about 10% of vandalism, about twice that of existing bots - now it's catching 50%-60%, in the range of the dataset trial). The false positive rate remains at the same 0.25% as before.

3130:

times before. Statistically, if a large portion of a user's previous edits have been vandalism, it's much more likely for their current edits to be vandalism. Alone this is not enough to trigger a vandalism classification, but it can push over the edge what might otherwise be a borderline edit. As the dataset grows, this will become more fine-tuned and less likely to be identified as vandalism, and the percentage of past edits that have been vandalism will remain a useful statistic in estimating good faith/bad intentions.

4927:

editors hit with false positives are newcomers with less than ten edits. If an experienced editor gets wrongly reverted, she presumably knows enough to take it with a grain of salt. But a good-faith user whose first or fifth contribution is reverted three seconds later by a bot is unlikely to return. Most don't bother to report the error. Of course, such harm must be balanced against our workload as vandal-fighters and the harm that might occur if more vandalism went undetected. I raised these issues at

5407: 4384: 572:

false positives no greater than an experienced, careful human editor would produce. 0.1% false positives is probably at least as accurate as most humans could be, and, at 40% of vandalism reverted (based on the discussion above) is still far more effective than existing bots. Given that ClueBot NG would be performing far more reversions in total than DASHBotAV and ClueBot have, setting the false positive rate below the 1% or 0.5% that has heretofore been considered acceptable is particularly important.

4014:

didn't really like the suggestion I saw somewhere that if two reviewers disagree then the edit will be dropped from the dataset - surely that is a recipe for blunting the sensitivity of the bot? If that is the case then I wonder what is happening to all those comments I've placed on difficult edits. What needs to happen is for those edits to be reviewed even more carefully, and perhaps even put up for community discussion. We'd all learn something, the reviewers as well as the bot. Enough!

870:

out by all the correctly classified ones, but they can affect calculation of threshold (by making false positive rate seem higher than it actually is). Also, at least one other edit of those 5 false positives would probably be caught by the post-processing filter (using a similar whitelist to existing bots) which will be added in production. Also keep in mind the bot is continually being improved. Many changes have been made since that report was generated. I'll post a new report soon.

5472:- A couple of users believed that the false positive report interface was too difficult to use practically. Then, one user actually took the time to find a false positive (stating he/she had to go through over 100 bot edits to find one), and tried to report it, determining that the interface was quite easy and painless to use. Users have also suggested some improvements to the interface, which we are now implementing. This discussion took place on the ClueBot NG talk page. 4072:

and allow them to view and add more comments, and change their existing vote. But the internal database currently cannot support this. The best way to implement this will be to wait until all edits currently in the database are classified (10,000ish), then upgrade the database. In the mean time, we'll see if there's any halfway point (possibly viewing controversial edits without being able to change the past vote) that we can implement without reconstructing the db.

4506: 3078:(2 diffs). Is it just detecting that he's restoring content that was reverted by someone else, or is there something about the edit itself that's triggering ClueBot? Also, the warning given on the user's talk page suggests that ClueBot NG was giving the final warning simply because of the addition of the word "an" ... does ClueBot by default only give a diff link for one diff, or is it actually only the second diff that triggered ClueBot NG? 6491: 921: 48: 2236:

accidental signatures classified as constructive, this type of thing won't happen. In addition to the signature, a possible complicating factor is that the bot can detect common vandal grammatical errors, such as unterminated sentences - and the user's edit, in this case, adds one. Again, as the dataset grows, and there are instances of where edits like this are not classified as vandalism, the bot will score these lower.

3478:

rolling back to a version last edited by a similar IP then it's worth digging deeper, and I've modified my anti-vandal tool to warn the user in this case. I seem to recall that at least one anti-vandal bot had a rule not to revert in such a case, so as not to 'lock in' an earlier bad edit. Perhaps what's really needed here is a semi-protected page the bot can write to flagging up articles it thinks need human attention.

6454: 866:"2100 vandalism edits reverted for every 5 false positives" would be accurate. However, there are more good edits on Knowledge than vandalism, so this isn't an accurate way to think about it. That's why I've been discussing false positive rate (which is a percentage of good edits), rather than the ratio, because the false positive rate is independent of the ratio of good edits to vandalism. 5331:"lack of appreciation" on the talk page because the user in that context was engaging in nonconstructive flaming and even making up quotes from the dev team to try to make us look bad. Nonconstructive complaining, and, worse, flaming, are not welcome at all. All other forms of comments and suggestions, even if we disagree with them, are welcome, and at the very least open to consideration. 5295:

vandalism. This is enough to get it noticed by human editors when they're beaten to a revert, but doesn't significantly decrease the time necessary for human patrollers to spend. Even with the lowered 0.1% false positive rate, ClueBot-NG is more than five times as effective as the old ClueBot, but the entire purpose of an antivandal bot should be to make a real difference.

5280:

core, but the final decision to revert may be overridden by some set metrics in the Knowledge interface. Because these metrics apply less-often to higher-scored edits, increasing the threshold lowers the percentage of would-be reverts stopped by the post-processing filters. Therefore, overall, even the percentage of reverts that are false positives has been decreased.

3987:

to scrap our entire existing dataset and replace it with the truly random sampling (and verified) edits from the review interface. But not enough edits have been reviewed yet to provide sufficient data for training. Is there anything we can do to make it easier to review edits, or make it seem more worthwhile to people? Thanks to those who are already helping!

5315:

choice to be made is simply how many false positives are acceptable, and the bot operates as well as it can, given that number. Ideally, with time, the FP rate can be decreased without hindering the bot's performance much or at all (as the dataset is improved), but false positives as a whole, and individual occurrences, can never be entirely eliminated.

5303:

operators spends most of his time working on dataset management, which as we've repeatedly stated, is what can most improve the bot - his remaining time is spent on real-life commitments. The third wasn't really involved in core development, and doesn't know enough about it to respond to false positives with anything more than a form-letter response.

5508:- The false positive rate is not calculated based on reported false positives (which may be less than the actual number). The false positive rate is calculated from a random sampling of human-verified edits, from the review interface, so it is accurate. Actually, actual false positive rate will be less than stated, due to post-processing filters. 903:, 8 of the 13 aren't even real false positives - they're edits misclassified in the dataset, so the bot is actually identifying these correctly. Since the whole dataset is human-reviewed, this demonstrates that the bot can, under some circumstances, be even more accurate than a human - it can even recognize and ignore errors in its training set. 5013:

in this area. However, if users do indeed contribute less time to vandalism reversion, the more likely outcome will be a reduction in their total contributions, since most users with a desire to write content are already doing so. Have many users actually said that they would write more for Knowledge, if only they weren't tied up with RC patrol?

5261:

that are probably vandalism, and could be used to improve human-assisted anti-vandalism programs. With sufficient refinement, the bot may one day be an acceptable fully automated anti-vandalism tool. No deprecation of your contributions is intended in the candid observation that the bot is not yet ready for mainspace live-editing approval.

2708:

Also as Cobi said, the false positive rate is not determined by false positive reports. We divide our dataset up into two parts - 2/3 to use for training and 1/3 for trialing. That 1/3 is run through the network and is used for rate calculations. This should be a very accurate way of calculating it, assuming a representative dataset.

5196:

current trial dataset would only yield a single false positive. I could run it with this, but I'd rather not, because there are some people who would interpret this as an inaccurate claim. Instead, as I have already explained (and you, once again, ignored), the threshold can be manually adjusted based on observed false positives.

3582:

of this). Also, I believe it's possible to get the bot accurate enough to the point where the previous edit would have already been caught it it were vandalism. Another thing to consider is that vandals tend to follow a pattern - if the current edit is reverted, it's likely previous edits in the same style would also be reverted.

2589:

are added to the dataset and classified as constructive, the bot will stop recognizing it as vandalism. But it seems to me that this kind of edit is very borderline - adding misspelled words is one things, but replacing correct words with misspelled ones, and formal words with informal ones, in multiple places, is another thing.

5753:, a perfectly fine insertion of a word for clarification. I have reverted back and intend to post on the IP talk page that it was a mistake. I realise it might be harder to distinguish between such edits and some forms of vandalism, but I imagine minor edits of this type are common and we don't want to deter casual editors. 5169:

adjustable and can be easily changed... there's no reason it shouldn't be approved", as though this would solve all of the bot's problems? Given the uncertainty you've described, I find it reasonable to evaluate the performance of the bot under its present configuration, without assuming that there necessarily is a better one.

4710:

exist, most are poor quality edits in some way (like the first of these two edits) that share traits with vandalism. There are occasionally unexpected reverts that don't appear to have any vandalism traits. This is a consequence of using a neural network as a core, and these should virtually disappear as the dataset grows.

1157:

and is being tested before restarting the process running the trial) and problems that can be solved by increasing dataset size. Please continue to report any issues so they can be added to the training dataset, and so I can add tags and other things (like redirects) that I may have missed to the special text handling code.

4898:

another user would be identified as vandalism. If so, the reversion should be suppressed. I assume that the bot would consider section blanking and replacement with "DONKEY BALLS" to be vandalism if done by a non-whitelisted user, as this is the sort of malicious edit that is most easily identified by anti-vandalism bots.

5257:

an unacceptably high (per-revert) level of false positives, then why is it making live edits at all? Continuing the dry run, and examining which edits it would have reverted, would provide adequate data on new false positives, while relieving the bot operators of the burden of responding on-wiki to false positive reports.

810:

vandalism, it's much less likely for the edit in question to be vandalism than if it's the user's first edit, or if the user has made 2 edits before, both of which were vandalism. (Also note that these scenarios do not alone positively identify an edit as vandalism. They only contribute to the result.)

251:. Because these algorithms must be trained on a dataset, there is also a convenient way to estimate accuracy before a live run - simply running the bot on a portion of its dataset not used for training. Currently, this is yielding a 60% to 70% vandalism detection rate - far above that of current bots. 5117:

this bot catches is missed. The benefits of using a neural network to allow human users to identify likely vandalism in a flood of other edits would be extraordinary, especially considering that the target false positive rate could safely be set much higher in a manually-confirmed reversion application.

5639:

Cluebot NG's performance is almost an order of magnitude better than all previous anti-vandal bots. Using novel algorithms and approaches, it truly is the next generation to practical automated vandal-fighting on Knowledge. And over time, as we continue to work on the bot, its accuracy will improve

5290:

To support this, take a look at one of the recent comments on the bot talk page, made after the threshold increase. The user states that they had to review over 100 diffs/reverts from Cluebot-NG to find a single false positive. While this isn't a wide sample set, it should give you some idea of the

5279:

The complaint was about the FP rate, not percentage of reverts that are false positives. The FP rate has been more than halved. In fact, the percentage of reverts that are false positives has also been decreased, due to the effect postprocessing has on the results. The FP rate is determined by the

5195:

About the ease of changing the false positive rate, it is exactly as I have described. The automatic threshold calculation is a helpful feature on top of the core. For excessively low false positive rates, it requires a very large trial dataset to accurately calculate. At 0.1% false positives, our

5012:

Your comments are incorrect, and self-contradictory. If users presently fighting vandalism reduce the amount of time spent on it to "contribute significantly more new material to the encyclopedia", the purported vandalism-reduction benefits of the bot will be blunted by the diminution of human effort

4623:

Also for whatever it is worth, the mechanism of how to report a false positive seems pretty daunting, espeically for a new user (I basically blew it off for example). I suspect that the average false positive will not report through the mechanism now required. So if your 0.25% is the reported false

4584:

It was not clear to me from the warning I got that I could revert the bot action. All the text was harsh, with none of the FAQ comments (addressed up page). Also, I was not sure if additional attempts to edit would make the bot treat me more and more as a vandal (as can happen with spam catchers on

4472:

The bot performs as expected. The false positive rate (which can still be adjusted if necessary) is set at 0.25%, which, after the revert exemptions, causes only a few false positives per day. This is below the false positive rate of existing bots. The vandalism catch rate, determined by using the

4258:

I support Rjwilmsi in his comments about the frequency of his bot's edits. Maybe there should be a few, but I get the impression that we're heading for a skewed dataset. The idea of "one day's edits" is flawed - much will depend on what bots are active that day, whether school's in or out, and what

4044:

Review interface is fine. Though it would be nice if 30% of edits for review were not my own bot's addition of persondata. To encourage wider participation include some stats (e.g. "after the n000 reviews bot accuracy has improved 5%" or whatever it is) and just politely spam the various tech village

4002:

I reply to this with some diffidence, because I've already talked enough on this page. (Thank you! Thank you!) But I do have a few comments. Firstly, please give us some time: quite a few people, me included, have already helped out, and I see no indication yet of contributors dropping out - rather

3649:

an excellent example of what I'd like the bot to hold off doing - or, at least, ask a human for help. Presumably the first bad edit wasn't bad enough, and the second was by an IP that had already been reverted on that page that day. So when it reverted the third bad edit it reverted to a version by

3581:

Interesting ideas. I'll leave it up to Cobi whether it's feasible or not to put additional rules to prevent reverting to a previous edit if the previous edit is potentially vandalism. I can say that this would likely incur a significant delay to fetch the extra information (although I'm not certain

3566:

Slightly separately, the idea of a page to log edits that are just below the threshold is attractive on the face of it, but in practice it may prove difficult to make it useful - many edits that are just below the threshold will be bad and will have been reverted by humans with good anti-vandal tools

3552:

If you have a good suggestion as to how to reliably determine when an IP is sufficiently similar to warrant reverting it as part of the rollback, I'd like to hear it, but I don't think that can be done without adding more false positives. As for the idea of a page for review, that could be done. Do

3537:

editor that this case needs to be trapped. It is frequently the case than when the same article has consecutive edits by different IPs that are in the same narrow range that in fact they were made by the same person. If the latest edit is vandalism then the earlier ones are suspect. Using rollback

3477:

is fine as far as it goes, but clearly needed to go one revision further back. I don't know whether the 100 or so things the neural network takes into account include the identity of the editor of the version that would be rolled back to. In my experience if reverting a bad edit by an IP would mean

3428:

Ooops! Just asking, but is this a case where the bot would have reverted itself back again? The word 'iincluding' is presumably rare in good edits. If a diff counts as vandalism both ways then surely it should hold off. Also, how much does the bot know about article categories? Words like 'love'

2910:

The method can be expanded to work with pretty much any namespace or content, but it should use a separate neural network, and must be trained on a training set from that namespace. I'd like to get the core perfected and approved for the main namespace first, then we'll look into generating datasets

2425:

This is the kind of false positive I'd expect - poor edits with borderline vandalism qualities. Even these should be reduced with a larger dataset (containing constructive edits with these traits). In addition to the word "your", the lack of space after the previous sentence was also a factor - it

2311:

This is purely a case of a gap in the dataset. The Bayesian classifier (ie, words) were not what caused it, alone, anyway. - "ball" isn't even in the Bayesian database (the bot learned that it occurs about equally in vandalism and nonvandalism). A few words may have contributed ("you" occurs in 548

1156:

So far, the false positive rate seems to fall right into the expected range of around 0.25%. It also seems to be reverting more than half of all vandalism on Knowledge, also as expected. The false positives that do exist seem to be primarily problems with redirects (which has been fixed in the code

927:

Approved for editing at 0.25% FP rate. 0.25% of edits means that on average, 3 out of 1000 edits will be reverted, which is lower than our current bots and many of our human editors as well. Crispy and Cobi and Tim are working continuously on this bot, and it should only improve from here. What more,

662:

Also, I should mention that the bot is constantly being improved every day. Accuracy will likely be even higher before the final trial. Anyone who feels they have something to add is welcomed to help. The more pertinent statistical information the neural network gets, the better. We've only added

5302:

It really is very time-consuming to respond to every false positive manually, and even with three bot operators, there's not enough spare time to go around. One of the bot operators has a wife, two jobs, and school to worry about, and still finds time to work on bot development. Another of the bot

5176:

approval. In light of the strong concerns many editors have expressed about the bot's excessive incorrect reversions, I don't believe that such a consensus exists. If the probability of an improvement in the bot's configuration is significant, then this request could be left open until it occurs, or

5149:

0.1% may be an acceptable value with decent performance, but the trial dataset is not currently large enough to accurately calculate the threshold. I will be able to accurately evaluate its effectiveness and calculate the threshold when the dataset from the review interface is approximately doubled

5116:

The value of integrating a neural network into an application like Huggle is that existing filters used to present possibly malicious edits for human examination are extremely primitive. I would venture to say that over half of the "filtered" edits are not vandalism, while much of the vandalism that

5085:

I am also surprised that, in all this complaining, nobody has suggested simply using an alternate false positive rate. I'll even take suggestions for thresholds. Every time Cluebot NG reverts an edit, it leaves a score. Suggestions for a score threshold or false positive rate (within reason) will

5066:

Whether or not the bot is used to revert vandalism is up to the BAG and will be decided at the completion of its trial. The benefits of using it in a human vandalism program are limited, as it is designed as a first line of defense. Considering that the false positive rate is adjustable and can be

4628:

0.25% false positives is based on a dataset trial with random, human-verified, edits. It is not only accurate, but the actual rate is less, because post-processing prevents reversions in some circumstances that will remove some false positives. The number of reported false positives has no bearing

4476:

While there are things that can still be improved to catch more vandalism, the false positive rate will always remain at a fixed percentage. Further improvements will yield a greater vandalism catch rate, but the false positive rate is adjusted by hand, and will not change unless it is decided that

4365:

Also, the classifications from the review interface are now enough to start being used. There aren't enough to use as a training dataset yet, but there are enough to use for trials. This means two things. First, it means we can train the bot using our entire existing dataset, instead of reserving

3986:

Looking at some of the current data from the review interface, it seems that our training dataset is significantly biased. The bot's current performance, while still better than existing bots, is significantly inferior to what it could be. This is due entirely to the bias in the dataset. I'd like

3875:

This is actually a real false positive. The dataset needs more instances of people reverting vandalism. Right now it has very few. The means used to generate it apparently don't generate a random sampling. As soon as the review interface generates a large enough dataset from random edit reviews,

3769:

Also, H3llkn0wz is right about the neural net - increasing dataset size and quality will both increase vandalism catch rate and decrease false positives. Cluebot-NG's false positive rate is very, very low, considering the sheer number of edits it reviews. Now, after fixing the programmatic issues,

3158:

The neural network functions by analyzing statistics. Because "an" is a common word, word-based statistics do not apply. What the neural network sees is a user inserting a short word into the middle of an article - a user than already has several warnings. Without the existing warnings, the score

1755:

One unique thing about using a neural network as the core detection engine is that it's a bit of a "black box". Sometimes it's not immediately apparent why an error occurred. Usually it's because the dataset just isn't large enough, and as it grows, these will disappear. Currently, the dataset is

1374:

The reports and charts are generated from the trial dataset. I can regenerate them (from the same trial dataset) after making these modifications, but it wouldn't look much different - the changes I'm making now are to prevent false positives from things which aren't well-represented in the dataset

1278:

The problem was not that it thought redirects were vandalism, but that there was no special handling for "REDIRECT". This has been fixed (special handling added). Testing the fixes right now, bot will be restarted with updates soon (it takes some time to retrain and test it). I also added special

770:

How do false positives affect the bot's learning ability? Will a higher false positive rate lead to "bad" entries in the dataset that could cause it to "learn" slower (or worse, get worse with time)? Or will the fact that its reverting more actual vandalism, including corner cases, mean that it will

586:

Keep in mind that number of reversions has no bearing on false positive percentage. False positives are measured based on how many legitimate edits it processes, not how many edits it reverts. False positives percentage is (number of legit edits marked as vandalism) / (total legit edits processed)

571:

It sounds like ClueBot NG could greatly outperform traditional Knowledge anti-vandalism bots, reverting significantly more vandalism with a lower false positive rate. One of the most important issues, however, is exactly where the acceptable false-positive rate is set. I would expect a percentage of

556:

Cluebot-NG does not use a "blacklist" of any sort. The words that compose an edit are taken into account in two ways; the presence in a set of predefined "word categories", and the result of a naive Bayes classifier (two, actually). The word categories (as opposed to a blacklist) allows the bot to

355:

It's difficult to perform a dry run on a userpage that adequately demonstrates the capabilities of the bot. The neural network takes into account not only the raw diff, but information on activity on the page, user activity, and other statistics. Also, the neural network is trained on a dataset of

6210:

I checked for that. Actually it was myspace replaced with facebook, so the bayesian score would be influenced by the addition of "facebook", but "facebook" isn't even in the bayesian database (close to 50/50 vandalism/constructive occurrences). There are a few words that are in the database, like

5635:

Controversy has sprung up, often due to misunderstandings about how various statistics are calculated and used. These have been clarified, and an FAQ page written to explain these issues. The remaining controversy has been addressed (false positive rate has been more than halved, report interface

5326:

Your suggestion about continuing a dry run is noted, but would never work in practice. Keep in mind that around or less than 1% (estimated as per the above-mentioned user that went through a series of diffs) of CBNG edits are now false positives. Reviewing data from a dry run would take 100x more

5322:

be able to be improved, more and more. It's a continual process. There will never be a point at which we can say "Stop. It's as good as it can be. Nothing more can be done." Just because there is still room for improvement doesn't mean the bot is not ready to make live edits. If this were so,

5256:

such that the cause of errors often cannot be ascertained and immediately corrected. Instead, false positives are generically attributed to "the dataset being too small". The only solution offered is to increase the size of the bot's dataset. But if the bot is not yet adequately configured to avoid

5214:

I have increased the threshold. With the new threshold, our trial dataset (containing 963 good edits) has zero false positives. So the false positive rate should now be approximately 0.1%. The catch rate has decreased a fair amount (it's hard to tell exactly how much, again due to dataset size),

4480:

The single most important thing for improving the bot is improving the dataset. Many people are already contributing large amounts of time to this purpose, and because of this, we can now use a real random sampling for statistics determination. As these people, and others, continue to help, we'll

4270:

Finally, how should we handle the case where the edit we're presented with is OK on it's own, but is the latest of a string of edits by the same editor that cumulatively are bad? In my experience, this is a common case when doing RCP. Normally, I'd hit revert, but for the cumulative edit. What's

4063:

For giving feedback, it's not really possible to set a certain goal, because it will always be improvable. It would work fine right now. Rjwilmsi's suggestion about giving statistics on the bot's current accuracy, given the current dataset, are definitely possible, though. We're going to work on

4013:

template or of "MRS FINKELSTEIN IS A GREAT BIG CHODE!!!", but in my experience some of these edits have proved quite difficult, and it's going to be the bot's ability to classify borderline cases correctly that will distinguish it from the rest, and justify the effort that goes into building it. I

3682:

Firstly, I think this bot is doing a great job, however, it is getting a large number of false positives. Some of these false positives are things which can easily be fixed, at the source code for the bot. Such as not reverting experienced users, ignoring discussion pages, not reverting CSD-tagging

3493:

There are a few things involved here in figuring out how to handle these situations. While it's true that if an edit is vandalism, immediately previous edits by the same user on the same article are probably also vandalism, it begs the question, why didn't the bot catch the earlier edit? The best

3451:

The word "iincluding" is not present in the dataset at all, so it would not contribute at all to the Bayesian score. If a word has never been seen before, it is not assumed to be good or bad, beyond a few basic things to detect if it's gibberish or leetspeak. Also, for a word to contribute to the

3297:

Ouch. This shouldn't be happening at all. The real issue can be fixed by enlarging the dataset (the current dataset doesn't contain many vandalism reversions) ... but there should be a hard threshold of edits per user. Users with more than 50 edits shouldn't be reverted at all - this is a bug in

3129:

The only concept the bot has of restoring old content is if the edit summary says so. In this case, the edit was probably identified as vandalism because it had borderline statistics (but would not ordinarily be considered vandalism), combined with the fact that the user had vandalized a number of

3030:

Thanks for your help. Although the bot currently is only being trained on mainspace articles, a few edits from other namespaces may have made their way into the random edit set. Classify these as you would normally (constructive, vandalism, skip). They simply won't be used for the main namespace

2819:

Cluebot NG is not meant to edit anything outside of the main namespace. This is apparently a misunderstanding between the developer of the core and the developer of the Knowledge interface. The interface will be changed to ignore edits not in the main namespace, unless at some point in the future

2707:

Just to clarify what Cobi said: He's correct about how false positive rate is determined. To accurately determine what it is during a live run, you have to count the number of false positives in a time period, and divide that by total number of legitimate edits that were made in that time period.

2550:

Thanks for pointing this out - this brings to my attention that, although links are removed (and analyzed separately) before being input to the neural net, total size difference includes the links. Because links are very rarely this long, this threw off the neural net. I'll look into modifying it

2540:

I corrected first the non-working external links pointing to Finnish Army Insignias on Finnish Defence Forces' website. Next i added a few spaces to the links' texts to correct their appearance and the bot reported this as unconstructive. I forgot to mention the last change as a small change, which

1815:

I too followed a link here to report a false positive, i really have no idea why it auto reverted my changes. The trigger that seems to have set off the bot was my use of the word "homosexual", although i was simply substitution it for the word "gay" in a sentence because it seemed more appropriate

805:

The bot does not automatically learn. If it did so, regardless of the false positive rate, its performance would simply remain status quo. Over time, we plan on manually growing the dataset, both by using human-reviewed random edits, and by adding specific false positives (and false negatives) to

611:

That is how false positive rates are measures - This is significantly better than existing bots nonetheless. Ie, if the existing bots are decommissioned and replaced with this, there will be fewer false positives, and much more vandalism caught. Also, vandalism comprises more than 1% of edits, so

6521:

Thanks. The false positive rate will remain at less than 0.1% for the foreseeable future, unless improvements are made to the bot which cause a slightly higher dropoff point than present, or the bot's accuracy improves to the point where it can be lowered without significantly affecting accuracy.

5310:

While the system for reporting false positives was criticized, no suggestions were offered on how to improve it (by the primary user doing the criticizing). Another user spent the time to find a false positive and report it, and not only determined that most of the criticism was invalid, but also

5306:

What's more is that individual responses are not necessary. As you point out, each one should be reviewed to make sure it's actually a false positive. And each one is. As stated in multiple places, each reported false positive is submitted to the review interface, where we can draw on community

5260:

I apologize if my critiques of the bot's operations, and those of my colleagues, seem inadequately appreciative of your software development efforts. The theory of the bot's operation is original and intellectually intriguing; the present code, configuration, and dataset can already identify edits

5104:. A member of BAG could presumably require you to reduce the rate. However, I have taken notice of the fact that, despite the mounting criticism of the bot's incorrect reversions, you have not actually reduced the false positive target. There would certainly be no objection to the bot running at a 4897:

Blanking a section of an article, and replacing it with "DONKEY BALLS" isn't even remotely acceptable behavior for a bot. It's also 100% preventable, without dataset extension: the bot simply needs to evaluate the reversion it is about to make, as though it were considering whether an edit made by

4341:

Cobi has made some major improvements to the review interface based on received comments. One of the important asked-for improvements is that users can now view what others have voted on edits they've already reviewed by clicking on the counter in the top-right corner, and potentially change your

4071:

For getting feedback on difficult edits, we've discussed ways to do this, and it may be possible to set something up, but it would likely require some restructuring of the database. The idea is to allow users to view a list of all edits they've classified, that others have classified differently,

4028:

About the reviewing interface, it is really easy to work with. I do regularly get an error message that makes me have to refresh. I get both generic error messages telling me something went wrong and I need to refresh, as well messages telling me that it is out of revisions. I like that there is a

3835:

Cluebot-NG does follow the same behavior of the old Cluebot in this regard - the interface to Knowledge (of which this functionality is a part) is largely just copied, and is the same code. Cobi knows the exact logic behind it, but my understanding is that, by default, it does not revert the same

3747:

I should point out that the false positive rate is selectable, and can be reconfigured at any time. I should also point out that the false positive rate is currently set at 0.25% - and that the actual number of false positives is *below* this. For the issues in the code, here's a quick overview:

3005:

Thanks - it's working now, and I've done a few. A few comments: If the dataset is aimed at mainspace aticles only, why was I offered a User talk space edit? I classified it as per its space - in an article it wouldn't've been good but it was fine as part of an attempt at dialogue. Also, some of

2965:

We're looking to double our current dataset size (currently a little over 30,000 edits) and replace it with a model closer to reality by using a truly random sampling of data. The interface is currently loaded with around 70,000 edits - about a day's worth. Each edit must be reviewed by at least

2588:

The bot may have been picking up on the direct replacements of formal terms with informal terms (ie. your replacement of "large" with "big"), and the replacements of words with incorrect spellings of the words (ie. your replacement of "undiscovered" with "indiscovered"). If enough edits like this

1922:

The neural network is only trained on articles in the main namespace. It is not (currently) meant to handle any other articles. I was unaware that articles from other namespaces were fed to the core. I'll tell the person running the interface code to exclude any edits not in the main namespace.

1084:

This also seems to be an issue with dataset completeness. There are no instances of complete page blanking in the dataset that are legitimate. As soon as these are added, this will correct itself. Also, users above a certain threshold number of edits should be ignored (and you should quality).

869:

On the other hand, a few other things to keep in mind: Of those 5 false positives, one isn't actually a false positive (it's a misclassified edit in the dataset). We are working to correct these few errors by manually reviewing edits. They do not affect training as the very few errors are washed

801:

How would I go about getting wider community input for the false positive rate? I agree with this, but so far there have been different acceptable estimates, but no real concensus. My recommendation is somewhere between 0.5% and 0.1%. Anything above 0.5% is probably too high, but close to 0.1%,

277:

Consider, for a moment: a false positive rate of 1.0% means that on average 1 in every 100 legitimate edits would be reverted as vandalism; 0.5% translates to 5 of every 1000 legitimate edits being reverted as vandalism. Keeping in mind that the enwiki edit rate is relatively high, false positives

5627:

The bot is performing well within its expected parameters. It was approved for Trial 1 for operation at 0.25% false positives, and it was always well within that limit. Halfway through Trial 2, it was changed to 0.1% false positives at user request, or 1 in 1000 incorrectly reverted edits (also

5330:

My comments about "lack of appreciation" on the user talk page discussion do not apply to you. While I disagree with you on most of the points you bring up (for reasons I believe are correct and stated above), it's also clear that you are trying to help, with nothing but good faith. I mentioned

5314:

The neural network is indeed a form of a "black box", but this is not the only reason that simply examining false positives will not directly and immediately help accuracy. As explained in multiple places, a certain number of false positives are absolutely necessary for the bot's operation. The

5275:

The comment you link to is mostly out of frustration. Despite your apparent surety that the bot is inadequate, you are one of only two people that I can see to strongly complain about the false positive rate, where many people have been happy and satisfied with it. I find myself making these FP

5191:

I believe BRFAs require consensus among the BAG. The reason the BAG was created is that non-members often don't have the knowledge or perspective to make informed decisions on automated processes. As neither you nor I are members, it's not up to either of us to decide, and we should leave it to

5153:

In lieu of a larger trial dataset, I can at present evaluate a given threshold, although as the bot changes, a set threshold can vary significantly. Reviewing the reported false positives (with a grain of salt - some of them aren't really false positives) may allow you to suggest a threshold. I

5141:

I have not lowered it below 0.25% because there has been no consensus. The BAG does have the final say, but if there was a community consensus, I would immediately adjust it. Looking at the bot talk page, there have been a number of instances of people happy with the bot's current performance.

4998:

keeping Knowledge twice as clean from undetected vandalism. I believe this is well-worth the minimal impact of less than 1 in 400 false positives, particularly considering that the warning makes it clear the revert may have been a false positive, and provides instructions for undoing the revert.

4600:

In this case, the false positive looks like it was partially a result of the earlier vandalism warning. Also, if you do not have any prior warnings, the first warning it gives is much nicer, and is more clear about what to do in the case of a false positive. If you have any ideas how to make it

4032:

About getting people to participate, this same problem is faced all the time by wikiprojects who organize 'drives' to improve certain parts of the encyclopedia. Some of the techniques I see used in these drives are: fixed timespans, clear goals, 'rewards' (meaning: glorified thank-you notes), and

3722:

Well that's kind of what I mean. I understand that the reviewed edits are either "good" or "bad". So if you have only reviewed bad edits, the bot is going to be more likely to assume that edits are bad. But I think it's gone to far towards the assuming the edits are bad. Maybe reviewing more good

3703:

That's how artificial neural networks work. In this case, it is basically a classifier - either vandalism or not, with a given certainty. If the neural network has never seen a given edit before, its internal weights are not trained to classify it, so it may end up giving an unexpected output. In

3209:

There are no (preset) weights. Statistics are combined using a neural network. To correct outlying datapoints like this, the datasize must grow. It's not really worthwhile to review the entire dry run - particularly since it's with an older version. The dataset review interface combines edits

2235:

The signature probably is the primary reason it was reverted - the training set doesn't include talk pages or areas where signatures are used, so without seeing a signature before, it probably seems like a random mashup of punctuation by a new user. As the dataset grows, and it sees instances of

1778:

the two and a half per every 1000 rollbacks by the bot that might seem to be indicated by the raw percentage.) If so, then as the dataset improves, the threshold for reversion will simply be lowered to continue to meet 0.25% target, resulting in more vandalism reverted, but new and exciting false

865:

About the comment on report.txt, you are reading it correctly, but the inference is a little off. The raw numbers of vandalism detected versus false positives are based on the trial dataset, which is about 50/50 vandalism/good edits. If Knowledge edits followed this same proportion, then saying

815:

A note about "penis" vandalism and similar: Currently, the bot's Bayesian database shows "penis" appearing in 157 vandalism edits, and 0 good edits. Because of this, a user simply adding the word "penis" alone to a page would likely be classified as vandalism. However, almost no legitimate edit

691:

Even its current performance is far superior to current bots. It's up to the BAG when the trial happens. As I said above, accuracy can only get higher, and development won't stop as soon as it goes live. We already have enough of a sample size (around 30,000 edits) to verify that it works very

5168:

If it's unclear whether any given reduced false positives target would retain sufficient performance to significantly decrease the rate of false positives per edits reverted (which is what the community is actually concerned about), then why did you claim that because "the false positive rate is

4926:

If you view at Knowledge purely as a data repository, that looks like great progress. However, Knowledge is also a community of editors, one constantly in need of "new blood". I believe that false positives do great harm to the Encyclopedia by driving away good-faith contributors. Most of the

4922:

After reviewing hundreds of bot edits, I'm concerned that the false positive rate may be set too high. The 0.25% false positive rate sounds impressive until you consider more intuitive measures of performance. Assuming 10% of edits are malicious and the bot reverts 60% of those, an 0.25% false

4724:

If you accept that 0.25% of edits it catches are FPs, then I think it needs constraints on how many times it will revert the same editor. If it's possible, 1RR would be advisable for edits that aren't certain vandalism. It can afford to be more aggressive on things like profanities and blanking.

4634:

If you have an idea how to improve the false positive reporting, go ahead and make any changes you feel could improve it. Just tacking on a new section was getting unmanageable. I don't have any particular preference on exactly how it works, but I think that actual discussion should be clearly

4489:

I'd like to ask for an extended trial. The bot is production ready, and performs much better than existing bots, both in terms of false positives and vandalism catch rate. But an extended trial will maintain interest in helping us to expand the dataset so it becomes as good as it can be, while

3355:

In my opinion, false positives of poor quality edits aren't quite as bad as false positives of good quality edits - but they still shouldn't happen. These should also be able to be prevented by expanding the dataset. The second of these two even looks like it's so poor quality that it could be

2935:

I already mentioned this, but it's important, so I'll bring it to attention again. Cluebot NG's accuracy depends almost entirely on its dataset. By fixing its current dataset, and helping to classify new edits, you can help to greatly improve its performance. We have an interface specifically

1682:

Edits like this are very difficult to distinguish from vandalism from a bot's point of view. The bot does specially handle things within quotes, but these lyrics were presented as normal page content, so they were handled normally. They contain unterminated sentences, slang, Bayesian keywords,

861:

It's very difficult to estimate the false positive rates of existing bots, because the majority of false positives probably are not even reported. Since existing bots are heuristics-based, they are frequently over-aggressive with many things that usually occur in vandalism, but can sometimes be

292:

I agree with this completely. One of the aims from the ground up has been to minimize false positives. The key is to realize that even humans can sometimes have false positives, particularly with the borderline edits (and this is the area where Cluebot-NG has a few false positives). Certainly

5357:

I realize that information and logic about behavior of the bot, particularly related to false positives, has been spread out over multiple discussions in different places. To make it easier to follow along, I have consolidated the information in a few places, and tried to explain it simply and

4997:

My position, and that of several others posting there, is that reducing human vandal fighter workload by half or more allows them to contribute significantly more new material to the encyclopedia. It also prevents half or more of the vandalism that currently gets through, from getting through,

4744:

The bot already adheres to 1RR. It does not revert the same user/article combination more than once in the same day. This allows users that are reverted as a false positive to simply redo their edit without being reverted. The bot does not contain simple heuristics, so we cannot make it more

4709:

False positives with Cluebot-NG are (essentially) inevitable. The amount of caught vandalism depends on a set false positive rate. Currently, the FP rate is set at 0.25% - this has generally been deemed an acceptable price for eliminating over 50% of vandalism. Of the false positives that do

4361:

In the last day or so we've made some major improvements with the dataset. We discovered an issue with the dataset we've been using. The output of the dataset downloader was not matching the output of the live downloader, essentially adding some degree of randomness to some of the fields, and

4168:

It may not be the best idea to make it clearly apparent that the bot will not re-revert the same edit. Even now, without this fact being made clear, a significant amount of vandalism is being caught, but slips through when the user re-vandalizes the page. This behavior of the bot is necessary

3773:

About the dataset ratio, this actually doesn't really matter. Having a dataset ratio that differs from reality will affect the average result score from the neural net, but remember that the threshold is calculated and calibrated using a set false positive rate, so even if the average score is

3607:

But your suggestions give me a few ideas for how to potentially improve accuracy. It may be possible to add an input to the neural network that is the time of the previous edit. Also, I may be able to add a parameter in cases where both the current and previous revisions are made by IPs - the

2697:

False positives is the percentage of good edits it classifies as bad. I.e., it classifies 25 out of every 10,000 good edits as bad. And, yes, we realize there needs to be work done to tweak it -- that is why we have a review interface so we can create a better dataset. We calculated 0.25% by

809:

The bot only looks at general statistics of the user's past edits (number of edits, time frame, number of past edits that were vandalism, number of unique pages edited). It does not process content of past edits. It learns things like, if the user has made two previous edits, and neither were

636:

become less frequent occurrences. I would suggest that setting the acceptable false positive rate as low as it possibly can be while still reverting a reasonable portion of vandalism increases the probability that the bot will be approved after its trial period (for the same reason, any "final"

3760:

Not reverting experienced users - Fixed. This was actually two separate problems. The first is that the edit threshold was initially too high, and has been decreased. The second is that the WP API was returning errors in a few cases, so the number of edits was being treated as zero. Error

3562:

The only other piece of information I can think of that a bot could go on is how recent the previous edit was. If the same article gets hit several times in a short period by several IPs in a narrow range, and one edit is clearly vandalism, then (in my experience) all those edits are suspect,

247:

Cluebot-NG is an attempt to revolutionize practical vandalism prevention on Knowledge. Existing anti-vandal bots use simple static heuristics, and as such, catch a relatively small portion of vandalism, and with an unacceptable false positive rate, many of which are likely not even reported.

5294:

You may wonder why I still disagree with such a low FP rate, even if I know that it increases overall "accuracy" - the reason is that, for an antivandal bot to really make a difference, it has to revert a significant portion of vandalism. Bots like the old ClueBot reverted an estimated 5% of

4266:

It would be nice to know what the criteria are for asking 'Are you sure?'. I've assumed that the answer is "This is the first time we've had that answer for this edit". If that's the case, then I'd say 0.5%-1% of the edits in the dataset are currently wrong classified. Is that right? Some

2494:

The bot most likely figured it was a poor/borderline edit based on statistics, and perhaps the word "estained". The previous warning for vandalism was used as an estimation of good faith (1/3 of all previous edits made were vandalism at the time of the edit). As with the other similar false

741:

That sort of false positive is very rare, and will be fixed over time as the dataset grows. Part of the algorithm uses a naive Bayes classifier on the inserted text. In the dataset we have, the word "banana" has been used in 4 vandalism edits, and 0 good edits. Since that user made no past

5338:

be room for improvement. The bot is already much improved from all predecessors, and only seems to be having more issues with false positives because of its much higher overall edit rate - so much so that things such as minor bugs in the Knowledge interface, that have remained unnoticed and

4259:

the major news item of the day is (we're getting a lot of stuff about the 2010 mid-term elections right now). If it has to be a random selection then it needs to sampled from a period of several weeks. Also, there should be a limit on the number of edits in the dataset by any given editor.

5233:

If the per edit examined false positive rate is halved, but the amount of vandalism caught is decreased by a similar factor, then the per-revert false positive rate, which is intuitively used to measure the accuracy of anti-vandalism work, will remain unchanged. Comments at this BRFA and at

5145:

Before you say that I do not listen to community input, you should take note of the fact that the original FP rate was set at 0.5%. I reduced it to 0.25% very early on, because at that point, there was consensus that 0.25% was preferable to 0.5%. I also evaluated your 0.1% suggestion, and

1703:

You should note that the account that made that edit seems to be a single purpose vandalism only account. So the number of past edits that were vandalism statistic would likely be higher. As such I'm not sure this specific edit is really a false positive due to the user's obvious bad faith.

5659:

We will never set the FP rate to anything above 0.25% (or 3 in 1000), and for now, it will remain at 0.1% (1 in 1000), as this is where community support lies. We will also always listen to the community and try to determine consensus if disagreement about the FP rate ever again arises.

5298:

You mention a recent complaint - but it is unrelated to the FP rate, or number of false positives at all. Rather, it is related to the handling of false positives. The discussion there clearly spells out our reasoning, and is mostly supported by at least one independent and impartial

4300:

Furthermore, about a "recuse" button, clearly, if you get an edit of your own, it's constructive. It's not a courtroom, just dataset generation. If someone who makes vandalized edits has access to the interface anyway, there are much larger problems than someone classifying their own

4064:

setting up a system to retrain and retrial the bot daily, each time using the new current dataset. The results of these trial runs will be posted. We may also be able to take this data over a period of a number of days and create things such as graphs of dataset size versus accuracy.

2270:

This is an instance where the dataset isn't large enough. For some reason, the only edits the bot has learned from with similar statistics have been vandalism. With a larger and more complete dataset, as is being generated now by volunteers, there will be fewer gaps in its training.

5253:, is that each and every false positive report is examined, and receives a response to determine whether it is genuine. When an edit filter produces an actual false positive, it is usually possible to modify it to prevent a recurrence. However, you state that this bot is a "black box" 4164:

It's probably a good idea to emphasize, "If this edit was made in good faith, do not be afraid to post a false positive report, and clear your good name." I can understand that new users could potentially be intimidated by a big warning, so something to this effect would probably be

4956:

Someone pointed out that many of the supposed "false positives" reported by the user(s) opposed to the bot's current performance are not actually false positives, and were indeed correctly reverted as vandalism. Even so, the counted number of false positives was within the expected

2019:. I wrote the core which does the main vandalism detection with the machine learning techniques. Cobi wrote the interface to Knowledge, which handles everything that's not machine-learning (exclusions, whitelists, etc). The interface was largely taken from the existing Cluebot. 830:

To me the false positive rate is the key question, since any vandalism reversion is useful. For comparison, what are the false positive rates of the existing anti-vandalism bots? IMHO any bot with a lower false positive rate than currently approved bots can be readily authorised.

1683:

abnormal use of capitalization, and other things which are fed into the neural network. It may be possible to add a special case for lyrics, but this might require that the training dataset contain several examples of lyrics being added. I'll look into the feasibility of this.

3804:

shouldn't be done – it's quite obvious they should have been. However it was my understanding that the old ClueBot would not revert the same thing twice. Was this just a coincidence or was that true? This bot doesn't seem to follow that same pattern. Is that intentional or not?

5960:

The edit by anonymous user 24.215.26.57 was perfectly legitimate. It appears that the bot was triggered by the facts that the edit was made by an unregistered user, combined with the added material being in full caps (a stylistic gaffe, to be sure, but one that can easily be

5016:

For the purpose of comparing this bot's false positive rate to that of human users, it is absolutely imperative that the rate be quoted in the same terms that would intuitively be used to measure the accuracy of human anti-vandalism efforts: the percentage of the total edits

1756:

large enough for the statistics mentioned above (about 60% of vandalism caught at 0.25% false positives), and just by estimating, it looks like these are approximately correct for the live run as well. As the dataset grows, these kinds of false positives will be eliminated.

3053:

I use Windows 7 and IE8. I had an edit come up in the review interface that caused IE to go into Compatibility View, and the diff it showed was blank. Sorry, can't remember which edit it was, but I marked the edit as 'Skip' (because I couldn't classify it) with a comment.

1850:.) Perhaps it might be advisable to add some edits to LGBT-related articles to the dataset of legitimate contributions, so the bot might (hopefully) be able to distinguish between references to actual homosexuality, and "gay" in the pejorative sense as a generalized insult. 4868:

It's not a race condition, just a normal false positive. It presently has a few issues with vandalism reverts, because there are few/none present in the dataset. This should stop with time when the review interface dataset becomes large enough to use as a training set.

4192:

Old topic, but I wouldn't put "and clear your good name", that implies that the reversion is saying something about their name in the first place. It might be good to compare it to a spam filter as well, since people understand that those sometimes have false positives.

4624:

positives, then the true rate is probably significantly higher. Some manual surveying ought to show that. (Still 0.25% may be right setting, but just realize actual collateral damage is higher). Of course, if I don't understand this, so be it...just trying to help.

4988:

The fact that ClueBot NG's false positives are not what one would expect from a normal bot and often are not triggered by things such as bad words, was reiterated. This makes it much clearer to users that are subject to false positives that they did not do something

4067:

For discarding edits where there's some disagreement, we've decided to change this to a scheme where every edit is always classified as something (Vandalism, Constructive, or Skip), and that the classification that is used must have at least 3x the votes as any other

5767:

This is one of those weird ANN things that will definitely be fixed by a larger dataset. The word "accidental" does not appear in the Bayesian database at all, so the statistical properties of the message must just be falling into a strange gap in the training set.

5558:

While the bot has generated some controversy, it has also received a large amount of support and praise - this support isn't on the BRFA, but may be useful. Only "pure support" message are included here - there are others that are part of controversial discussions.

5343:, more than noticeable by vandal fighters and other users alike, as clearly evidenced by numerous comments on the user page and talk page. There are no significant outstanding problems - particularly when significance of problems is compared with previous AV bots. 278:

tend to add up. I'm not sure what the magic number is going to be, but my suggestion would be that you should at least keep in mind the idea that as a whole the community seems to prefer dealing with vandalism manually if it avoids labeling new users as vandals. --

6284:

The edit you're referring to looks like vandalism to me. It's not obvious vandalism, but removing all mention of "Romania" from an article where it is pertinent usually constitutes vandalism. The bot is capable of catching non-obvious vandalism such as this.

1793:

It can be adjusted to whatever people want. As the dataset improves, I'll update the graphs. The 0.25% was determined by looking for a sharp dropoff point below 0.5% on the graph. As the dataset improves, the false positive rate will be lowered as well.

2981:

Is there any, umm, help or documentation for this interface? I've activated my Google account, I've got as far as the screen that asks for my Knowledge username to match my Google email id, and I'm looking at a page that says "Stored.". Now what do I do?

1446:

Actually, the parameter is number of times user has been warned. This is one factor out of over a hundred, and alone, is not sufficient to indicate that an edit is vandalism. You can think of it more as an "estimation of good faith" for borderline edits.

5199:

As I already stated, I'd much rather spend my time actually working on improvements, as it's a continual process, instead of repeating myself and arguing. Whether or not the bot is approved in its current state is up to the BAG as soon as the trial ends.

2765:(now deleted) reverted the blanking of a page by its author. It is very confusing for an author who realises his page is inappropriate and blanks it, which is a frequent occurrence, when the inappropriate page is restored in stead of being tagged db-g7. 663:

the things that we can think of off the tops of our heads. All of this information is configured at run time using configuration files, so new metrics can be easily added. If you have something to add or suggest, stop by irc.cluenet.org #cluebotng .

6211:"rumors", "very", and "happy", but none of these have particularly high scores - definitely shouldn't be high enough to cause a false positive. There must just be some statistical property of the edit that fell into a gap in the ANN's training set. 4169:(unless the false positives are eventually somehow reduced to an incredibly low amount), but making it apparent to vandals, and even providing links for them to re-vandalize in one click, could drastically reduce the actual effectiveness of the bot. 899:, there's a sharp dropoff after about 0.25% false positives. At 0.25% false positives, the score threshold is 95.4%, and 63.7% of vandalism is caught. In reality, the number of false positives will be less than demonstrated here. Looking at the 2132:

I should also note that, while Cluebot-NG has a false positive rate comparable to some humans (if a human were to review every single edit made to Knowledge), the false positives are not always the same ones that you might expect a human to make.

1816:

in the context, and created a better flow in the prose. Oddly the Bot also reverted a number of other minor changes i had made, which were mealy filling in missing words, and could in no way be construed as vandalism. Here are the changes i made

1551:"ClueBot NG produces very few false positives, but it does happen. If you believe the change you made should not have been detected as unconstructive, please report it here, remove this warning from your talk page, and then make the edit again." 1379:

for details), but have not gotten any volunteers yet. If we get enough volunteers, then we'll add all the edits the bot has seen to the review queue, and generate a separate trial dataset from this (then, we can generate the stats and graphs).

384:

Capped at 0.1%, it currently catches about 40% of vandalism. We're working to improve this by identifying under what circumstances edits are falsely identified as vandalism and correcting them. You can view one recent trial report in whole at

5466:- A couple of users believed that the 0.25% max. false positive rate (at most 1 in 400 false positives) was too high, with a fair amount of debating. Eventually, at one of the user's suggestion, the false positive rate was lowered to 0.1% max. 5123:- and the fact that you made it shows a disregard for community input. Whether there's a reason the bot shouldn't be approved depends largely on whether you are willing to respond to the community's critiques by lower the false positive target 4254:

I'm sure that at least twice now I've had the same edit come up for review twice - one on the safety of microwave ovens and one about common given names in Azerbaijan. Does the interface not check whether a reviewer has seen the edit before?

1028:

Reverting itself has been fixed. The other errors are due to the dataset not being broad enough (adding these edits to the dataset and retraining should rectify this) and REDIRECT not being in the list of wiki markup (being fixed right now).

4262:

I could do with three more choices: "This needs a subject matter expert", "Content dispute", and "Recuse". I've had an edit of my own come up, and Rjwilmsi has been in the uncomfortable position of having to classify an edit by his own bot.

452:(apparently, any use of the words "sex" or "pussy" in an edit by a non-whitelisted user is sufficient to trigger reversion), provided that the existing anti-vandal bots are decommissioned. Will this the algorithms of this new bot adhere to 4342:

own vote in retrospect. Note that you cannot view what others have voted before voting yourself - this is to prevent any prior bias. Also, the logic to determine the final result has changed, and contested edits are no longer discarded.

596:

Are you saying that if it processes 100,000 edits, 100 of which are vandalism, it will revert 70 of the vandalism edits and 500 legitimate edits? (given 70% catch rate at 0.5% false positive). If so, that's far, far worse than I thought.

2659:

The 0.25% false positive statistic doesn't seem correct; if you're calculating it by taking the number of people who take the time to post on this page divided by its total amount of edits, you're going to get a very skewed "statistic".

774:

A somewhat WP:BEANS-y question - If it looks at users' past edits, is it possible to game the bot? If a user makes a legitimate edit to an article about penises, will that make the bot less likely to detect "penis vandalism" by the same

6195:

I thought I fixed it, but seemingly not. There's no other reason this edit should have been reverted. To fix it before, I just removed all instances of "i.e." before processing. I just changed it to instead replace it with "Ie".

4745:

aggressive for certain offenses. However, it may be possible to override the 1RR (this rule does make the bot miss a fair amount of second-time vandalism) in some strict circumstances, such as where the edit has a very high score,

4297:"Subject matter expert" should be able to be handled by the skip or the refresh button. "Recuse" is handled by refreshing. "Content dispute" should be a skip. We could make a dedicated button to do the same as refreshing, though. 3016:). Couldn't these have been automatically classified as OK? Finally, if there's any disagreement with another reviewer about any of my classifications then I'd appreciate learning about it, if only to improve my own performance. 5517:

Those not already familiar with how the bot works should read these links. They are critical to understanding its behavior. These were written during Trial 2 in response to numerous repeated questions for the same information.

5070:

I find it unfortunate that I have to spend so much time repeating over and over things I have already said, when I could be spending the time improving the code. But apparently this is a necessity in getting community approval.

4601:

clear (on subsequent warnings) that the bot will not revert the edit twice, without making it obvious to vandals (and easy to revandalism), we'd love to hear it - we've been trying to think of a good solution to this ourselves.

460:

because they contain "bad" words? (though questionable language may legitimately be one of the factors which weighs in favour of reversion.) Also, blanking of content is sometimes legitimate; automated restoration of copyright

1779:

positives to replace the ones that have been eliminated. This is why, in the discussion above, I suggested that a 0.1% false positives target would be more conducive to community acceptance of the bot, and ultimate approval.

2683:. By my (very informal) data, ClueBot NG has a 4% false positive rate. Don't get me wrong, the bot is unique and the work you're doing is great, but the bot definitely needs some tweaking before being let loose unmonitored. 4366:

a portion for trials. This should slightly increase accuracy. Second, it means that the statistics we give about the accuracy of the bot are now guaranteed to be accurate and unbiased (the 50%-60% above is an example).

310:

I know that this discussion is focused more on eliminating false positives rather than increasing ClueBot NG's accuracy, but I've noticed that the bot seems to miss content-removal vandalism more often than the other bots.

5112:

you to do so is worrisome. Therefore, I am evaluating the bot based on its present mode of operation, rather than some hypothetical alternate configuration that might exist, had you been more responsive to the community's

4108:

be false positives; it will never be possible to recreate 100% of the brain of a human RC patroller in computer software. When users who make acceptable edits have them reverted, misunderstandings arise. For example, see

3538:

in this case runs the risk that earlier vandalism may become locked in - further vandalisms will show up, to bots and in anti-vandalism tools, as bad diffs, but the earlier vandalism might remain in place for some time.

1525:

The following is the log entry regarding this warning: Four (energy drink) was changed by 109.46.144.246 (u) (t) ANN scored at 0.817485 on 2010-11-18T07:44:16+00:00 . Thank you. ClueBot NG (talk) 07:44, 18 November 2010

3687:

that edits are vandalism. Saying "we need to expand the dataset so the bot picks up more vandalism" makes sense, saying "we need to expand the dataset so the bot picks up less false positives" doesn't, for me anyway. -

3770:

it's only getting a few false positives a day. As I mentioned earlier, Cluebot-NG's false positive rate is very low, but the false positives it does have aren't necessarily the same ones you'd expect from another bot.

2838:. I've removed everything from the optin list, for now. Keep in mind, when users add pages there, they are inviting the bot somewhere where it has not been tested or designed for. It may work well. It may not. -- 536:

is an administrator with nearly 150,000 edits, the identification of this edit for reversion suggests that a whitelisting mechanism is not (currently) implemented. Is this a planned feature, perhaps through the use of

4543:

It means the bot incorrectly identified the edit as vandalism. This false positive looks like it was caused by the bot not recognizing HTML color codes as such. This will be fixed as soon as the parser is complete.

5021:

that are false positives. Once the false positive rate is provided in a comprehensible format, I believe that the ugly truth will become apparent: one would be hard pressed to find nearly as many false positives in

2390:

Occasionally it has issues dealing with song lyrics because they do not follow standard acceptable wiki formatting. We're looking into adding special cases in code, and increasing dataset size should help as well.

2312:

vandalism articles, and 45 good articles), but this should have been counterbalanced by other words ("22" occurs in 82 good articles, and 22 vandalism articles). With an increase in dataset size, this should stop.

4949:

Initially, several people misunderstood the meaning of "False Positive Rate", although it has been clearly explained in multiple places that is means "portion of legitimate edits that are incorrectly classified as

3737:

Right now the dataset is roughly 50/50 vandalism/constructive. The dataset we are generating with the interface will come from a day's worth of edits (roughly 70k edits), and will have a more realistic ratio. --

1627:

Sorry for the confusion - that page is currently "left-over" from the original ClueBot. We thought it would be good to keep false positives here while the BRFA is open, so all reviewers can get a good idea of its

6397:

Since the user did not add a space or comma, it was treated as a very long nonsense word. A full parser may help this, as could a very extensive dataset including examples of this kind of well-intentioned error.

4473:

random sampling of edits from the review interface, is right around 55%, about an order of magnitude more than existing bots. This puts a very large dent in vandalism on Knowledge, and will continue to improve.

1308:

page, which is currently incorrect, and I replaced it with the track listing cited in the reference, yet it was reverted again. My edits should stop being reverted as they are not unconstructive nor vandalistic.

329:

Very interesting. The last 2 anti-vandal bots performed dry runs on their user pages before being approved. This does not require prior permission, AFAIK. And just curious, have you test your program against the

6084:

This is mostly an issue with dataset completeness (not many/any instances of incorrectly added templates), but it probably would not have been classified as vandalism if that IP had not vandalized in the past.

4852:

restored a bit of vandalism that had just been reverted in the same second. My guess is that it appropriately identified the vandalism but missed that it had been changed before it go to do it itself? --John

4103:

ClueBot NG seems to catch vandalism much better than the old ClueBot did. However, we must not forget that regardless of how amazingly well such statistical techniques as artificial neural networks work, there

2940:. You need a Google account to use it, and we need to authorize you to access it. If you'd like to help out, please follow the link and go to the signup section. Help is needed, and greatly appreciated! 6067: 6061: 5443:

False positive rate was lowered from the previous 0.25% (as it was for Trial 1) to 0.1%, at user request, more than halving the number of false positives. The change was made about half-way through Trial

2152:-- the content of the page was "test" so I tagged as {{db-g2}} and was reverted by ClueBot NG. There's no way that should have happened. I've never had any bot revert any of my speedy taggings previously. 626:

In any case, at a 0.1% false positive rate and 40% of vandalism reverted, ClueBot NG represents an opportunity for significantly improved performance, so that more vandalism is reverted, while events like

4749:

more than half of the user's previous edits have been vandalism, or something like that. But before overriding 1RR under any circumstances, there should be significant community discussion on the issue.

1504:

I added some variations to the information for Kings, the drinking game. It actually has a section for variations and it had kicked off the suggestions I have added. Any suggestions how to make it stick?

5939:

I seem to have gotten credit for several vandal edits from this IP address on subjects which I have never written about-footballers and the Fritzl family and so on. The edit on "The Sparrow" was legit.--

5063:

The statements you say are incorrect are not my saying. They are summarized from the talk page discussion. Please take the time to read there. And yes, there are users that have made these statements.

4138:

If the user believes that his edit is not vandalism, he may repeat the edit, and the bot will not take action. (Include instructions for reverting the bot using undo, maybe even a link in the talk page

4129:

The bot is not perfect, and it will never reach 100% accuracy, although its false positive rate has been set to revert only 1 in 400 legitimate edits. This is to help Knowledge remain free of vandalism.

4970:

Much of what is explained on the bot's user page and on this BRFA was reiterated, including how the FP rate is calculated, and how a certain number of FPs are necessary for the bot's proper operation.

2118:

Note: Cluebot-NG has reviewed over 70,000 edits so far, resulting in a handful of false positives, which are either being fixed now programmatically, or will be fixed with the growing of the dataset.

2166:

This seems to be an issue in some cases because the current dataset does not contain instances of speedy deletion tags being added. We are generating a new dataset now which should solve the issue.

2698:

training with 20,000 edits in our current 30,000 edit dataset, and then having it classify the remaining 10,000, and seeing how many it said are vandalism, when our dataset said they were good. --

1847:. One thing that ought to be apparent from the false positives up till now is the bot's persistent homophobia - any mention of "gay" or "homosexual" seems to be enough to trigger it (for instance, 6260:

The bot was treating what was inside there as a quote, and text inside quotes isn't processed (other than counted). Combined with the multiple other recent warnings, it was enough to trigger it.

4960:

A user used the bot's administrator shut-off (intended to be used when the bot is behaving unexpectedly) to stop the bot's operation. The same user later reversed this decision about a day later.

1507:

tried to edit "Hoy_Me_Voy" to change sentences to a more appropriate tone, but ClueBot flagged the change as vandalism, put a notice on my talk page, and reverted the change back. What do I do? (

4463:

Vandalism catch rate at approx. 55%. Vandalism revert rate at an estimated 40%. Not all caught vandalism is reverted, mostly because the bot won't re-revert edits, and users often re-vandalize.

6278:

That what they do is vandalism. The Vlachs are not Romanians, they are ethnic group recognized by the Constitution! In census their language is recognized as Vlach! So, we must to respect that.

5807:

who has previously vandalized. As SMS is the only topic relating to the definition that follows the SML/Texting entry ("Idc" or "I.D.C." is an acronym for "I don't care.") I made the change to

5648:

The developers request that the bot be approved to operate at a false positive rate of the operators' discretion. We would like the ability to adjust the false positive rate for a few reasons:

4650:

I'm not sure mechanically how to improve it. It's just discouraging for a user to feel like he is prey to a machine and then that the appeals process is arduous. No biggie, just a datapoint...

509:

constitute vandalism, human judgement is required to determine whether this is actually the case; the content may have blatantly violated the sourcing or neutrality requirements of the policy.)

5686:

for more information about why this happens, and why it is necessary. Reports posted here are reviewed by the bot developers in case anything can be done to the bot to improve its accuracy.

2996:

Sorry, I need to fix that message to be more intuitive. It means you were added to the list of users for admins to review. I've approved you. You should be getting an e-mail about it. --

1864:

Yeah, this is what needs to happen. In the current dataset, there are no instances of these words being used correctly. As soon as these edits are added, this problem should correct itself.

3836:

user/article combination twice in the same day, with some exceptions. These exceptions are for the article of the day (which this is), and any articles listed in the "angry opt-in list".

3751:

Redirects - Fixed. At the beginning of the trial, there was no metric to recognize these to input to the neural net, so the neural net just saw it as shouting. This metric has been added.

3312:

It's OK, I'm not offended. Well, not much. Strangely, I've just been presented in the review interface with one of my own reversions. I added a comment asking for a 'Recuse' button ...

2350:

This appears to be because the bot was counting "i" and "e" both as uncapitalized sentences, and "i" as an uncapitalized 'I'. Thanks for pointing out this special case. It is now fixed.

5041:

The time spent on developing this bot shouldn't be considered wasted, however. Perhaps the neural network feature could be integrated with a human-assisted anti-vandalism program such as

1514:

tried to make a change on Fernando_Garibay, adding "so happy i could die" to his 2010 productions but it got removed. the proofs in his website and its already a source so what do i do.

5655:

Stated FP rate is less than actual FP rate due to post-processing filters. As these post-processing filters are modified, the core FP rate may need to be modified to maintain accuracy.

307:

Update - I ran the dataset without any prior user information and accuracy was almost the same, with only a slight dip of about 4%. There was no significant effect on false positives.

5450:

False positive reporting switched from freeform reporting to the old ClueBot false positive reporting interface, so we can more easily use the data from reports to improve the dataset.

4923:

positive rate implies that 3.6% of the bot's reverts (1 in 28) are false positives. If the bot makes 2,500 reverts per day, that's 2,410 good reverts and 90 false positives per day.

5142:

Additionally, at least one user has explicitly stated that they are happy with the aggressiveness. Without consensus, the only sane option is to delegate to the BAG for arbitration.

1611:

to give a false positive report, but as far as I can tell, there's not actually a section set aside here for that purpose. I apologize for any intrusion in my making this section. —

5242:, came in after the threshold for reversion was increased. The system for reporting false positives, and lack of individualized responses, was also critiqued. The claim that (with 3410:

Both of these can only be explained by the dataset not being large enough. I'm not really sure why the second one was misclassified - it must just be a hole in the training data.

5598: 846:

If I understood the info from the report.txt that's linked to above, if the bot catches 2100 vandalism edits while having 5 false positives it sounds like it's doing very well.

5608: 5588: 5254: 5121: 5276:

rate changes, just because it's very time consuming to carry on these debates about the same topic, where the pertinent information has already been stated in various places.

2966:

two different people (more if the first two disagree). If we can get this data, I believe the bot's performance can significantly improve, even from what it's at right now.

1741:(it's not immediately clear to me why the edit might have seemed like vandalism at all; the bot is surely not policing the addition of unreferenced material to Knowledge.) 404:

Thanks, that's just what I was looking for. It does look like the sweet spot is 0.5% at this point. I would feel more comfortable with a 0.1% false positive rate though.

4620:

WRT the bot, I would leave it as is, in terms of harsh remarks for tagged vandals. The collateral damage is probably small and the benefits high. Just keep an eye on it.

3194:

BTW, I tried to get myself a Google id to help out with reviewing the dataset and ended up writing a scathing comment about the user hostility of the application process.

4132:

There are certain types of edits that the Knowledge community does not find acceptable. (Summarize the vandalism policy here, including the different types of vandalism.)

3913: 5613: 4280: 1580: 5323:

the bot would never be ready. A point has to be set at which the FP rate is considered acceptable - you've suggested the FP rate of 0.1%, and that has been acted upon.

4703: 4678: 4045:

pump, bot owner noticeboard, huggle talk pages etc. Can we establish how many reviews are needed to reach production-level accuracy, set a target for the review phase?

2175: 2161: 6205: 5892:

Bot reverted an edit where I added the relevant quote to a reference. Quote was buried in lengthy source text therefore quote in reference was necessary & useful

5447:

Data from dataset review interface has grown in size enough to use as a trial set, and more accurately calculate the threshold and statistics from false positive rate.

4111: 4023: 3659: 3617: 3503: 3461: 3365: 3321: 3219: 3063: 3040: 2991: 2920: 2905: 2598: 2435: 2280: 1304:, since it is an upcoming release. Yet it was since reverted & I got a vandalism warning. Same thing happened when I corrected the UK edition track listing in the 5379: 5352: 5270: 5209: 5186: 5163: 5136: 5095: 5080: 5058: 4719: 3869: 2400: 1803: 1788: 1765: 1288: 1148: 1094: 1055: 1038: 1023: 646: 581: 566: 4822: 4772:

this is reverting the addition of an internal English Knowledge link, formatted as an external link. Presumably it looks like spam, but perhaps this can be tweaked.

4610: 4351: 1692: 4759: 4739: 4240: 4185: 3851: 2504: 2469: 2359: 1172:

It would be better if you assign a special page for reporting FPs instead of here. Also, what is the number of edits above which a registered user will be ignored.

6220: 5593: 5224: 4151: 3949: 3446: 3403: 3385: 3350: 3276: 2583: 2420: 2265: 2096: 1893: 1713: 6152: 5007: 3932: 3885: 3307: 3168: 3153: 2829: 2385: 2245: 2028: 2010: 1978: 1964: 1950: 1932: 1195: 1181: 365: 5334:

The upshot of this all is, the bot is already in a much-more-than-adequate state to be running live. There is indeed still room for improvement, but there will

4333: 4319: 4088: 3972: 3732: 3717: 3576: 3557: 3025: 3000: 2675: 1727: 879: 825: 794: 269:

I think one of the main concerns about anti-vandal bots is not about how good they are at detecting vandalism; rather, it's above all about how good they are at

5578: 4953:

Using estimations from several users on actual number of false positives, the actual live false positive rate was calculated to be well within the stated 0.25%.

4831:

to the level 1 template. I will see about fixing this properly in the code, as it cannot be done in a template, but it is somewhat low-priority right now. --

4776: 4653: 4246: 4161:

Several of these points are already mentioned in the warning the bot posts on user talk pages (although it can't hurt to have it be elsewhere in a FAQ as well).

4039: 2887: 2878: 2524: 2489: 2455: 2345: 2321: 2110: 1351: 1337: 751: 701: 686: 591: 248:

Cluebot-NG shares no code with the original Cluebot, and uses completely different algorithms to detect vandalism. Details of these algorithms can be found at

5108:

false positive rate than the one under which it was approved for the trials. This refusal to modify a clearly problematic bot task until a BAG member actually

4907: 3845: 3547: 3419: 3258: 2789: 2692: 2669: 2646: 2632: 2623: 2560: 2207: 1917: 856: 841: 6531: 5583: 5030:. Any edits which can be automatically identified as almost certainly vandalism, without an unacceptably high false positive rate, are already blocked by the 4055: 3292: 2230: 302: 6562: 6255: 5928: 5603: 5573: 4835: 4202: 4007: 3783: 3697: 3487: 3139: 3010: 2911:

for other namespaces. If necessary, while getting it to work with other namespaces, the old heuristics-based cluebot could be run just on those namespaces.

2842: 2774: 2717: 2702: 2059: 1655: 1456: 1397: 1389: 1007: 714: 621: 529: 427: 413: 399: 379: 350: 6317:

I simply added the Latin text of the memorial underneath the section heading "Legionares Memorial" The text is from note2 source, this is not "vandalism".

6190: 6120: 5861: 4892: 4878: 4807: 3742: 2856: 2306: 6362: 6176: 5948: 4616:

I think the inital vandal tagging was a bit of over-reaction for the type of error I committed, but I don't want to get into it more. I'm not "wounded".

3528: 2545: 2055:. Not warning users was due to someone setting the bot's shutoff page, and due to a bug that has now been fixed, it only honored that page for warns. -- 1602: 1241: 1122: 1070: 975: 6341: 5994: 5881: 5722: 4594: 4172:

The old Cluebot has a nice user-friendly false positive reporting mechanism. When Cluebot-NG goes into production, we'll bring this interface live again.

4098: 3203: 1677: 1369: 736: 320: 6139:

There's a change of "facebook" to "myspace", and that might have been a contributing factor, since I presume vandals sometimes spam social site links. —

6079: 5970: 2740: 1859: 1750: 1632: 1566: 1542: 1487: 1441: 891:

Examining the bot's current performance, I've decided to recommend a false positive rate of 0.25%. I also posted the report from a recent dataset trial

6294: 6269: 6134: 6094: 5834: 5820: 5777: 5762: 5736: 5727:

A few of the other false positives above have also been related to interwiki links - it's a dataset issue. As the dataset grows, this should be fixed.

4644: 3723:

edits would deal with this - I don't know. Basically, it's seems too concentrated on identifying bad edits, and not enough on identifying good edits. -

2757: 1640: 1620: 6468:

Either the page is no longer relevant or consensus on its purpose has become unclear. To revive discussion, seek broader input via a forum such as the

3210:

randomly from several sources - one of these sources is edits that bot is unsure of. So the dataset review interface is by far the best way to help.

3124: 606: 550: 518: 6052: 5427: 4979:

The statistic that "1 in 400 incorrectly reverted legitimate edits is worth 200 in 400 correctly reverted vandalism edits" was put forth, and debated.

2142: 2127: 6411:

Definitely fixed by larger dataset. With the current dataset, all mass removals of links have been vandalism. Clearly this is not always the case.

6023: 5652:

We select an appropriate rate based on generated graphs of statistical performance, looking for a dropoff point, which can change as the bot changes.

1273: 1078: 5619:

It's also worth noting that this praise is coming from people who are familiar and used to the old ClueBot, so they are noticing a real difference.

4976:

It was implied that all users subject to a false positive will leave Knowledge and never edit again. This was given without proof and is incorrect.

3996: 1418:, which I reverted. When the user made the same edit again, it was reverted by ClueBot NG. Does the bot assume that reverted edits are vandalism? -- 1014:

for no apparent reason (or perhaps the means of identifying vandalism is so problematic that the bot really is marking its own edits as vandalism.)

912: 672: 6043:

which seems like a legitimate addition to the Languages list for the article. (I don't read this language, but I was able to locate the article.)

5849: 4460:

False positive rate below the set 0.25% (the false positive threshold is calculated before applying revert exemptions, such as minimum edit count).

3243: 1590: 6306:

There was no vandalism. This is an automated process, so not sure why my edit was tagged and reverted. I wanted to keep working on the article!

5154:

would be open to running the bot for a day or two with a given set threshold within reason, to see if its performance in that mode is acceptable.

4940: 3757:

Non-main namespace pages - Fixed. This wasn't actually a bug, but was due to importing the old Cluebot's opt-in list. It has since been cleared.

3677: 2192: 1166: 1001: 5919:

should have been reverted. The IP was simply trying to say that the actress is now in another soap and it got reverted as possible vandalism. --

1826: 6239: 4786:

Hi - it seems that ClueBot NG makes a new header for the month for every first warning they give out - there are 3 November 2010 headers here :

2975: 2949: 1360:

Is it possible to generate new false positive vs catch rate charts once the bot has a few days under its belt so that we can see the progress?

1216: 5672: 5249:

is troubling. The accepted Knowledge standard for responses to claimed errors in automated tools designed to stop malicious edits, as shown at

5067:

easily changed (I don't know why I have to keep saying this, people just don't seem to understand), there's no reason it shouldn't be approved.

5023: 1343: 1318: 1310: 286: 183:

The core is written in C++ by Christopher Breneman. The interface to Knowledge is written in PHP by Naomi. The dataset is maintained by Tim.

5951:

Also will someone please tell Jimmy wales the multimillionaire megalomaniac philanthropist to take his smirkung begging face off every page !

2813: 1427: 634: 462: 370:

What is the accuracy when the false positive rate is capped at 0.1%? Is there a chart somewhere of the catch rate vs false positive setting?

5250: 5027: 4574: 4499: 4375: 6011: 5663:

After approval, we will restart the bot, so it can continue doing its job of keeping Knowledge clean, and reducing vandal-fighter workload.

5339:

unreported for the three years the original ClueBot has been running, are now being noticed and fixed very rapidly. Even in trial, CBNG is

4827:

It's an issue with the fact that ClueBot NG (and ClueBot) simply append a subst'd template to the end of the talk page. Someone decided to

3829: 3188:

appears in the Dry Run but does not seem to be clear vandalism to me. Maybe greater weight needs to be given to the context of the change?

948: 2149: 4781: 2762: 2637:

I can't believe I missed that - wow. Yeah, it's the same problem that has caused a few other issues. Not enough reverts in the dataset.

1437:

is number of times reverted. So, not directly, but the bot probably did pick up on the fact that that user had been reverted before. --

692:

well. This sample size is what the other figures in this discussion are based on. The larger the dataset, the more accurate it will be.

4519: 4029:

counter in the corner, so one can set a goal for themselves as 'I will review x amount of revisions this session', and then do just that.

3766:

Even context-specific false positives are at a much lower rate than existing bots, and can continue to be improved with a larger dataset.

1718:

Thanks for pointing this out. Yes, the bot does take this into account. It may be a factor into why this particular edit was reverted.

4862: 4553: 4225: 3469: 6403: 2801: 1938: 5751: 4579: 3608:"distance" between the IPs. This parameter would just be the smallest CIDR subnet size that contains both IPs. I'll look into this. 3508:

I thought ClueBot used rollback anyway, but that's not important. It's not the same editor, so that's probably why it didn't analyze.

335: 631: 628: 450: 447: 5487:

These are clarifications on some things are are available elsewhere, but are restated here because they are commonly misunderstood.

3225: 215:

Current statistics indicate approximately 70% of vandalism is caught, so it would be editing approximately 70% of vandalized pages.

6515: 5287:. Observed FP rate is likely to be significantly below this, as many FPs are caught and eliminated by the post-processing filters. 4510:

Approved for extended trial (14 days). Please provide a link to the relevant contributions and/or diffs when the trial is complete.

3801: 3191:

Is it actually worthwhile for humans to review the whole of the Dry Run? If so, what's the best way to flag what's been reviewed?

90: 4967:

based on number of reported false positives, but is an accurate number based on a dry trial of random edits not used for training.

3180: 1464: 4973:

The impact of vandalism and importance of human vandal fighters' time was reiterated by myself and several other impartial users.

3430: 1994: 6382:

Sole addition of two (non-consecutive) exclamation points. Might be fixed by improved dataset, but very rare edge case anyway.

5034:. Contributions which are accepted by the filter currently require human judgment to evaluate, to avoid automated violations of 3283:

Exactly the false positive I was coming to report. I reverted ClueBot's reversion. Curious to see if I get a warning too. :D

3098: 3075: 1591:

http://en.wikipedia.org/search/?title=Kamen_Rider_OOO_%28character%29&curid=28242801&diff=396938724&oldid=396938713

5677: 4770: 4310:

For an edit that isn't vandalism, but is in the same string of edits where vandalism occurred by the same user, just hit skip.

3981: 2609: 2444: 2373: 464:

or BLP violations is particularly disruptive. I suggest that blanking in the following situations never be reverted by a bot:

3754:

Various tags - Fixed. A metric was added for certain tags, and template names are now removed before statistical processing.

2153: 1594: 4917: 3104:

Feel free to move this up to Section 2 if it fits better ... I'm not meaning to make my edit stand out from all the others.

1905: 1872: 1571:

Okay, it was in fact another user that re-reverted the change, and not ClueBot. Please disregard that follow-up comment. --

5318:

Extending this, it can be seen that your following comment about the bot not yet being ready is invalid. The dataset will

2221:

was made by an inexperienced user, and just seems to be reverted by accidentally adding his signature ~~~~ to his edit. --

1665: 1407: 983: 965: 21: 5239: 3857: 2728: 2681: 1411: 925:

Approved for trial (14 days). Please provide a link to the relevant contributions and/or diffs when the trial is complete.

6424: 6354: 6168: 5940: 5246:

bot operators) everyone responsible for the bot is simply too busy to articulate responses to the false positive reports

4928: 2513: 2377: 1572: 1558: 1534: 1515: 1479: 1249: 1130: 1010: 5238:

suggest that this continued high level of inaccuracy would still be unsatisfactory. The latest complaint about the bot,

5120:

Your claim that "in all this complaining, nobody has suggested simply using an alternate false positive rate" is untrue

4564: 2569: 992:

Thanks for pointing this out - "REDIRECT" was not included in the list of wiki markup to ignore. We are adding it now.

105: 6469: 5986: 5873: 5714: 4849: 4801: 4586: 3647: 2538: 723:

by 68.229.109.100 (talk · contribs) at 2010-10-27 19:37:08 - ANN scored at 0.956627) was a false positive, however. :P

5283:

I'd also like to point out that, due to these post-processing filters, the given FP rates, whether 0.25% or 0.1%, are

3795: 3006:

the edits were edits made by approved bots that might equally well have been made by a human (e.g. RjwilmsiBot adding

2930: 273:

marking legitimate edits as vandalism. Therefore, one main concern from the document might stem from the following:

259: 6071: 5247: 5230: 4158:

A FAQ like this would indeed be useful, and I'll work on writing something up. A few comments on your list, though:

4843: 3231: 2883:

I copied all of the control pages from ClueBot's userspace. I forgot to remove all but the comment at the top. --

2795: 2409: 6404:

http://en.wikipedia.org/search/?title=Harry_Potter_and_the_Deathly_Hallows_(film)&diff=prev&oldid=398006837

5683: 5682:

Less than 0.1% of constructive or well-intentioned edits are misclassified as vandalism by Cluebot-NG. Please see

5531: 5363: 3426: 1848: 1044: 100: 4992:

It was suggested by an impartial user that his/her own human false positive rate is likely greater than the bot's.

4524: 1983:

I am confused by what you mean by "developer of the Knowledge interface"? Exclusion compliant means following the

953: 69:

That link should provide an easier and more pleasant experience than editing the table at the bottom of this page.

6567: 4635:

separated from false positive reports, and the false positive reports should be represented in a concise manner.

4294:

I don't see why there should be a limit, so long as it is proportional to the number of edits they make in a day.

1770:

Do you intend keep the target false positive rate at 0.25%? (for editors new to this discussion, that's 0.25% of

678:

When do you think you will be ready for a trial and what kind of sample size will you need to verify the method?

5985:

was a misconception, as mute swans do not "sing right before their deaths". The bot reverted the one-word edit.

4356: 2804:

too many false positives on the wikipedia science reference desk. and the error report ID fucntion seems broken.

498:

The blanking only removes content (doesn't replace it with something else) on an article that was previously in

339: 5367: 2214: 1834: 928:

with the dataset being improved, FP rate is actually lower than stated, so this should be an allright FP rate.

443:

I would support an anti-vandal bot which could avoid absurd false positives such as these recent reversions by

63: 1817: 1324:

The issue with redirects and disambiguation pages is known and is being corrected. Cluebot-NG has not edited

767:

I think it would be good to get some wider community input on what an acceptable false positive rate would be.

6346:

False positive due to Bayesian keyword. Will be mitigated by complete parser and fixed by improved dataset.

6125:

I don't see any reason this edit would be classified as vandalism. It must be a dataset completeness issue.

3335: 2084: 1647: 331: 135: 5432: 4075:

Continue to try to figure out the few bugs that are causing random (but harmless) occasional error messages.

2495:

positives here, increasing dataset size and including cases where previous vandals make constructive edits.

6431:

the section with valid text. Your edit removed the addition and restored the typo corrected by the editor.

6040: 4883:

Thanks for the reply. I'll assume you will take care of getting it into your database if useful? --John (

4218: 4142:

The bot operators are open to suggestions of how to improve the bot, including reports of false positives.

2679: 1471: 494:

The blanking only removes content that contains no references in a form the bot can recognize (<ref: -->

3533:

Perhaps I didn't make myself clear. It's precisely because rollback only rolls back consecutive edits by

2075: 5872:

I just added a note that QuickTime is required to hear the Start-up and Chimes Of Death in MacTracker. --

5539: 5359: 4963:

There was some misunderstanding about the accuracy of the false positive rate. 0.25% false positives is

4684: 2369: 2292:, while not referenced is definitely not vandelism. Not sure why it was labelled as such (use of ball?). 2254: 1937:

One thing this has brought to attention, is that the exclusion compliance is apparently not working, see

1845: 1739: 1230: 1112: 120: 5229:

I certainly hope that the change in the threshold for reversion ultimately produces better results than

4559: 4392: 3374: 3332: 2657: 1533:; ClueBot flagged my change as vandalism and reverted my change to the previously vandalized version. -- 110: 6248: 6109: 5916: 5535: 3774:

higher in general, the threshold will also be calculated to be higher, and will normalize the results.

3475: 3434: 3118: 3092: 2750: 2614:

Considering that the article is about the TLD and not the slang word, this edit seems very borderline.

2290: 1325: 1305: 1301: 538: 6379:

Correcting name of a band from "Wakey Wakey" to "Wakey!Wakey!" but the system thinks it knows best...

4982:

Whether or not the time that human vandal fighters spend patrolling edits is significant, was debated.

4534: 3704:

fact, the network needs much more good edits in its set than bad edits to not make false positives. —

3265: 2330: 1608: 5924: 5825:

The dataset had not previously seen "SMS" or "SML" in all caps, and as such, treated it as shouting.

5311:

did indeed give some suggestions, which are being discussed and will likely be implemented very soon.

5235: 2896:

Then I'm concerned. Can you at least consider widening the scope to include the Template namespace?

2157: 1598: 1415: 1297: 1296:

I edited the Greatest Hits So Far redirecting page to a disambiguation page in order to include both

1260: 130: 5901:

Added a new link to the recently-released English version of the product website, but was reverted.

4212: 2529:

The bot is failing to recognize this as a link. It currently recognizes external links with either

1871:

Cluebot seems to be immediately reverting all contributions made by IP's to the Reference Desk, eg.

125: 6186: 6116: 5858: 5800: 5792: 4570: 4329: 4276: 4019: 3655: 3572: 3543: 3483: 3442: 3399: 3381: 3346: 3317: 3272: 3199: 3059: 3021: 2987: 2936:

designed for this, and should make it easy for volunteers to help out. The interface can be found

2901: 2835: 2579: 2416: 2261: 1659: 1508: 1401: 2862: 719:

Generally pretty impressive, imo. I will say that the fourth link I clicked on in a list of 2500 (

6358: 6172: 5944: 5804: 5421: 5266: 5182: 5132: 5054: 4903: 4696: 4671: 4397:

The trial is now over, and I'd like to take a moment to go over what was found during the trial.

3909: 2874: 2381: 1855: 1784: 1746: 1576: 1562: 1549:

Okay, after this happened the first time, an automated message appeared on my talk page stating,

1538: 1519: 1500:

It's pretty obvious the article was written by someone working for the producer of the medicine.

1483: 1269: 1245: 1126: 1074: 1051: 1019: 979: 707: 642: 577: 546: 514: 499: 6394:

Looks like the user was adding the "Smoky Bacon" flavor of Pringles chips and was nailed for it

5086:

be considered, and I can post stats on bot effectiveness given either the threshold or FP rate.

3392: 3069: 1668:). The edit needed to be reverted due to copyright issues, but it was not an act of vandalism. — 1009:

is a rather inexplicable reversion. Also, the bot seems to have a bad habit of reverting itself

6337: 6014:

one triggered the bot. It was a pretty clear similar addition to what was already on the page.

5990: 5877: 5718: 4659: 4590: 2478: 2411:

I don't like the use of the word 'your', but to call the original edit vandalism is a stretch.

1955:

I'll tell the developer of the Knowledge interface. It handles all whitelists and exclusions.

1673: 1636: 1616: 115: 6181:

Just asking, but has the problem of "i.e." being treated as ultra-short sentences been fixed?

6066:

My edit was misconstrued as nonconstructive. I was trying to put up a cleanup tag on the page

2480:

Maybe it was a poor, unsourced edit to content about a living person, but it's not vandalism.

6243: 6075: 5966: 5459:

Several controversies not (conspicuously) present during Trial 1 were raised during Trial 2.

5031: 4734: 2747: 1204: 1105: 900: 896: 637:

trial, either dry or with actual reversions, should be conducted with whitelisting enabled.)

612:

although that example does show how the rates are calculated, it does not represent reality.

5527: 5127:. (I still believe that my suggestion of 0.1% above is prudent.) The ball is in your court. 4304:"Are you sure?" comes up in some circumstances where the current bot isn't sure on the edit. 389: 6527: 6290: 6265: 6216: 6201: 6130: 6090: 5920: 5830: 5816: 5773: 5758: 5732: 5668: 5375: 5348: 5220: 5205: 5159: 5091: 5076: 5003: 4874: 4818: 4755: 4715: 4640: 4606: 4549: 4495: 4371: 4347: 4251:

N.B. Bot owners - feel free to move this to a new page if you feel it doesn't belong here.

4236: 4181: 4084: 3992: 3968: 3881: 3841: 3779: 3613: 3499: 3457: 3415: 3361: 3303: 3254: 3215: 3164: 3135: 3036: 2971: 2945: 2916: 2825: 2785: 2713: 2688: 2665: 2642: 2619: 2594: 2556: 2500: 2465: 2431: 2396: 2355: 2317: 2276: 2241: 2203: 2171: 2138: 2123: 2106: 2024: 1960: 1928: 1822: 1799: 1761: 1723: 1688: 1452: 1385: 1333: 1284: 1237: 1191: 1162: 1144: 1118: 1090: 1034: 1012: 997: 971: 908: 875: 821: 747: 697: 668: 617: 562: 423: 395: 361: 298: 178: 95: 4033:

advertisement on places such as the 'Community bulletin board' (on the community portal).

2426:

registers punctuation present in the middle of words (other than things like apostrophe).

8: 6544: 6182: 6148: 6112: 5853: 5546: 4936: 4566: 4325: 4288:

You should not have gotten the exact same edit twice (maybe the user re-did their edit?).

4272: 4114:

User talk:ClueBot Commons/Archives/2010/November#are anons not allowed to post subst:prod

4015: 3865: 3728: 3713: 3693: 3651: 3568: 3539: 3479: 3438: 3395: 3377: 3342: 3313: 3288: 3268: 3195: 3149: 3055: 3017: 2983: 2897: 2852: 2809: 2575: 2412: 2257: 2226: 2006: 1974: 1946: 1913: 1186:

Users with more than 50 edits are now ignored. This is what the old ClueBot was set at.

1177: 346: 40: 5568: 5563: 3940:"This is probably a silly question, but what does the "NG" stand for? New Generation? -- 1203:

I changed a stub article, flagged as poor into a redirect and got a false positive. See

6461: 6453: 6048: 5416: 5262: 5178: 5128: 5050: 4899: 4690: 4665: 4046: 3893: 2870: 2866: 2446:(filling in the chronology part of an infobox with a link) definitely isn't vandalism. 2183:

This is probably a silly question, but what does the "NG" stand for? New Generation? --

1900: 1851: 1830: 1780: 1742: 1709: 1265: 1212: 1047: 1015: 847: 832: 785: 638: 573: 542: 510: 155: 75: 4985:

Whether or not human vandal fighters catch 100% of vandalism immediately, was debated.

3376:

Not vandalism. Not enough good edits in the database with the word 'toilets', right?

3186: 720: 453: 6333: 6019: 4813:

This is a known issue and appears to be intermittent. Cobi is working on fixing it.

4037: 2303: 2090: 1888: 1669: 1651: 1612: 1347: 1314: 734: 275:"Estimates given to me for an acceptable false positive rate range from 1% to 0.5%. " 4135:

The bot's revert of a user's edit does not necessarily mean that it is unacceptable.

1111:

on a page with a broken redirect (to a page that has been deleted) and was reverted.

525: 6484: 5962: 5042: 4726: 4516: 4222: 4143: 3945: 3926: 3822: 3239: 2770: 2629: 2516: 2481: 2447: 2337: 2188: 1423: 316: 2869:? No one requested that ClueBot NG revert pages outside of the article namespace. 2332:

looks like a revert of a perfectly legitimate and correct edit (see definition of

1328:, it did not consider these edits vandalism. They were reverted by another user. 6523: 6507: 6417: 6388: 6373: 6326: 6286: 6261: 6232: 6212: 6197: 6126: 6102: 6086: 6033: 6004: 5909: 5842: 5826: 5812: 5769: 5754: 5744: 5728: 5664: 5523: 5371: 5344: 5216: 5201: 5155: 5087: 5072: 4999: 4870: 4814: 4751: 4711: 4636: 4602: 4545: 4491: 4367: 4343: 4232: 4177: 4080: 3988: 3964: 3900: 3877: 3837: 3775: 3609: 3495: 3453: 3411: 3357: 3299: 3250: 3211: 3160: 3131: 3032: 2967: 2941: 2912: 2821: 2781: 2709: 2684: 2661: 2638: 2615: 2590: 2552: 2496: 2461: 2427: 2392: 2351: 2313: 2272: 2237: 2199: 2167: 2134: 2119: 2102: 2039: 2020: 1956: 1924: 1795: 1757: 1719: 1684: 1448: 1381: 1376: 1329: 1280: 1187: 1158: 1140: 1086: 1030: 993: 904: 871: 817: 743: 693: 664: 613: 558: 533: 485: 472: 419: 391: 357: 294: 147: 76: 5028:

the same number of edits reverted by this bot during approximately the same time

4787: 3249:

This can be fixed by enlarging the dataset, and by fine-tuning word categories.

6141: 5808: 5796: 5599:

User_talk:ClueBot_Commons/Archives/2010/November#ClueBot_NG.2C_you_dazzle_me.21

5035: 4932: 4198: 4060:

Interesting points. The following have been added to Cobi's and my TODO list:

3861: 3724: 3706: 3689: 3284: 3145: 3111: 3085: 2848: 2805: 2736: 2222: 2218: 1999: 1970: 1942: 1909: 1365: 1173: 602: 409: 375: 342: 17: 4539:

What does possible vandalism by 41.252.6.218 to version by SicaSunny Means ??

6556: 6301: 6044: 5609:

User_talk:ClueBot_Commons/Archives/2010/November#Four_second_vandalism_revert

5589:

User_talk:ClueBot_Commons/Archives/2010/November#Now_I_really_am_impressed.21

4931:

and got many interesting replies, and now I am moving the discussion here. --

4888: 4858: 3341:

Poor quality edits to poor quality article rather than deliberate vandalism.

2749:

the improvement of a poor quote translation was reverted within seconds (see

2049: 1987: 1908:, maybe keep it to article space, or at least out of the discussion space? - 1705: 1530: 1208: 782: 503: 219: 2959: 2955: 6015: 4946:

To summarize the discussion on the talk page (not in chronological order):

4796: 4267:

feedback on how many errors the reviewers have uncovered would be welcome.

4034: 3517: 2674:

Just looked over the last 50 of the bot's contribs (for the record, that's

2293: 2079: 1875: 937: 892: 724: 680: 444: 151: 5799:. SML documentation provides no mention of IDC, and the person who added 5146:

determined that it would cause a significant drop in the bot's catch rate.

1226: 386: 4513: 4481:

eventually be able to use the random sampling as a training set as well.

4291:

We will be able to add a more random sampling over a span of a few weeks.

3941: 3921: 3806: 3235: 2766: 2542: 2184: 1419: 706:

I ran it for a few hours today in dry run mode and exported it's data to

312: 254: 249: 187: 4505: 6502: 6111:

reads like a content dispute where the bot should not be taking sides.

5215:

but it should still be at least twice as effective as the old Cluebot.

4791: 4773: 2333: 1475: 1139:

Thanks, will also add this tag to markup list and edit to the dataset.

279: 5614:

User_talk:ClueBot_Commons/Archives/2010/November#Trial_over_already.3F

5049:

vandalism, with a user-adjustable threshold score for identification.

2937: 5982: 4832: 4316: 4194: 3739: 3554: 3106: 3080: 2997: 2884: 2839: 2732: 2699: 2056: 2016: 1438: 1361: 1222: 711: 598: 588: 405: 371: 198:

Vandalism detection and reverting using machine learning algorithms.

5038:

and discouragement of new editors by an unacceptably inaccurate bot.

2751:

http://www.nybooks.com/books/imprints/classics/the-way-of-the-world/

1262:. In the absence of a better solution, I suggest that article =: --> 4884: 4854: 4490:

still reverting vandalism just as well as it would in production.

3891:

Just kind of a random question, but, what does the "NG" stand for?

47: 5100:

I am well aware that the target false positive rate is adjustable

4828: 4434:

A few "bad words" that haven't been seen to be used in good edits.

3144:

Could you also clarify why the user was warned for adding "an"? -

3856:

IP added vandalism, another IP removed it and ClueBot tagged the

3553:

you just want it for any previous IP who is in the same /24? --

3511: 2074:

ClueBot NG reverted a speedy deletion tag db-vandalism which was

931: 6537:

The above discussion is preserved as an archive of the debate.

6070:, and not having enought experience doing that, I did it wrong. 5594:

User_talk:ClueBot_Commons/Archives/2010/November#I.27m_impressed

3567:

almost immediately, so it'll be out-of-date almost immediately.

2628:

But the bot actually reinstated the piece about the slang term.

2460:

Definitely something that could be fixed with a larger dataset.

6315:

Re Tropaeum Traiani cluebot NG revert 10.18am 20 November 2010

2015:

The bot's code is created primarily by two people - myself and

1493: 6543:

To request review of this BRFA, please start a new section at

710:-- warning, this is a very large page (about 2500 links). -- 192:

See Christopher Breneman for access to subversion repository.

39:

To request review of this BRFA, please start a new section at

5579:

User_talk:ClueBot_Commons/Archives/2010/November#Well_done.21

4426:

Outstanding issues that can be fixed by improving the dataset

4307:

We are working on adding some more feedback to the interface.

3523: 1969:

Also, are you aware that it's not currently warning users? -

1557:

and placed another warning on my talk page. What is this? --

1264:

article conversions be removed from the "vandalism" dataset.

943: 6408:

Dude, WTF? You restored an every-word-has-a-link vandalism.

6165: 6161: 1221:

The robot is automatically reverting my edition on the page

388:. Among other things, it includes the graph you requested: 5584:

User_talk:ClueBot_Commons/Archives/2010/November#Impressive

4512:

It seems the biggest thing needed is the improved dataset.

2820:

we train separate neural networks for separate namespaces.

2802:

User_talk:ClueBot_Commons#Cluebot_-too_many_false_positives

338:, and if you have, how it has performed in comparison with 5604:

User_talk:ClueBot_Commons/Archives/2010/November#Wow.21.21

5574:

User_talk:ClueBot_Commons/Archives/2010/November#Good_Work

468:

The blanking edit adds a copyright violation template, or

6353:

The stupid bot deleted my entry! MAKE IT STOP, MOM!!!

2376:), however my changes were reverted as vandalism... why? 1434: 165: 62:

If you have come here to report a false positive, please

5024:

250 edits reverted by an experienced, skilled human user

4125:

if we are to approve this bot. We need to explain that:

2611:

Edit incorrectly reverted for not being 'constructive'.

1593:

How would it even know if this is vandalism? Stupid bot.

5848:

Bot reverted edit that wasn't vandalism, but fancruft.

5713:

The bot undid my interwiki link for the Template page.

2368:

I made some improvements to the English translation on

418:

I'll update the graphs as the bot improves and learns.

6332:

I was adding a source that is already in the article!

3298:

the Knowledge interface code. We'll correct it ASAP.

2731:

was reverted within seconds for no obvious reason. --

1492:

An attempt to remove the blatant advertising from the

1069:

I blanked an attack page and got a warning. Not good.

5851:

I reverted the bot edit to let the editors decide. --

3394:

Certainly not vandalism; an improvement if anything.

2958:! We've added some stats for who's doing what right 2551:

into removing links in a preprocessing step instead.

530:

Knowledge:Bots/Requests for approval/ClueBot NG/Trial

6460:

This page is currently inactive and is retained for

4418:

Incorrect downloading of some fields in the dataset.

4121:

I believe that a concise, informative FAQ page is a

3074:

ClueBot NG gave a final warning to an IP editor for

2571:

Teach your bot what wikify and Manual of Style are.

2780:We'll add an exemption for the author of the page. 1529:I attempted to revert an instance of vandalism to 202:Links to relevant discussions (where appropriate): 1478:that was reverted, despite being a genuine edit. 964:I fixed a botched attempt at a redirect by a noob 6554: 3919:Believe I read it stands for "Next Generation". 1225:but "Ideologia" could refer either to the album 33:The following discussion is an archived debate. 4231:Looks like a dataset completeness issue to me. 6367:Not a false positive. The edit is vandalism. 5240:User_talk:ClueBot_Commons#Problem_with_the_bot 3429:and 'hate' and 'pregnant' are normal in, say, 2753:for a source of the correct quote translation) 1046:appears to be an incorrect reversion as well. 5789: 5784: 5251:Knowledge:Edit filter/False positives/Reports 4929:User_talk:ClueBot_Commons#false_positive_rate 4789:- if you can fix it that would be great :) -- 1774:; the number of incorrect reversions will be 495:, a reference template, or an external link.) 6563:Approved Knowledge bot requests for approval 5750:The bot undid an IP user's good faith edit, 5364:Detailed Information on CBNG False Positives 5045:, as a means of identifying edits which are 3876:we'll replace our current dataset entirely. 1844:I've taken the liberty of reverting the bot 6068:List of Representatives from North Carolina 6062:List of Representatives from North Carolina 5494:- The false positive rate is calculated as 967:and got reverted by this bot. Thanks, Jon. 5981:I was pointing out that the legend of the 3860:as vandalism. Timing issue I'm guessing? 2956:people helping with dataset classification 1553:I did this, and it re-reverted the change 4401:Problems found and fixed during the trial 6242:edit doesn't look like vandalism to me. 1085:We'll look into lowering the threshold. 5706:Reason/explanation/fix/discussion etc. 3852:Another false positive for your dataset 3431:Category:Serial drama television series 2574:What sort of a word is 'indiscovered'? 14: 6555: 6497:to operate at operators' discretion. 4437:A few random, rare statistical flukes. 3761:handling has been added to solve this. 3650:an editor it had previously reverted. 57: 4654:ClueBot NG Reverting good faith edits 4247:Some comments on the review mechanism 2541:may have affected in the bot report. 1433:Well, one of the parameters into the 6448: 4431:Reverting occasional vandal reverts. 3267:You'll understand I'm a bit miffed. 3234:didn't seem like vandalism to me. -- 2256:Can't see why this revert was made. 5496:Number of Incorrect Classifications 1233:. The robot needs some adjustment. 802:there's a dropoff in effectiveness. 456:, and refrain from reverting edits 213:Estimated number of pages affected: 27: 5476:Commenting on Every False Positive 4099:Providing a better user experience 1279:handling for disambiguation tags. 1006:That's not the only error so far. 28: 6579: 6238:Nothing to do with me really but 5684:Information About False Positives 2758:Reverting page deletion by author 2035:Exclusion compliance fixed. The 45:The result of the discussion was 6489: 6452: 6167:. Trying to improve the page. -- 5470:Ease of False Positive Reporting 5405: 4504: 4382: 3963:Our intent was Next Generation. 2198:Our intent was Next Generation. 919: 164:00:35, Monday October 25, 2010 ( 46: 5506:False Positive Rate Calculation 5413:We'll post a summary shortly. 5341:making a significant difference 1993:template in this case, such as 1738:Here's another false positive: 172:Automatic or Manually assisted: 5492:Meaning of False Positive Rate 4450:Larger, more accurate dataset. 2863:populate User:ClueBot NG/Optin 13: 1: 6363:22:31, 20 November 2010 (UTC) 6342:21:43, 20 November 2010 (UTC) 6295:16:06, 18 November 2010 (UTC) 6270:21:30, 16 November 2010 (UTC) 6256:20:04, 16 November 2010 (UTC) 6221:12:54, 16 November 2010 (UTC) 6206:12:37, 16 November 2010 (UTC) 6191:10:50, 16 November 2010 (UTC) 6177:08:34, 16 November 2010 (UTC) 6153:12:50, 16 November 2010 (UTC) 6135:12:46, 16 November 2010 (UTC) 6121:08:32, 16 November 2010 (UTC) 6095:02:08, 16 November 2010 (UTC) 6080:01:52, 16 November 2010 (UTC) 6053:00:16, 16 November 2010 (UTC) 6024:23:07, 15 November 2010 (UTC) 5995:03:01, 20 November 2010 (UTC) 5971:16:37, 19 November 2010 (UTC) 5949:16:09, 19 November 2010 (UTC) 5929:09:38, 19 November 2010 (UTC) 5882:22:53, 18 November 2010 (UTC) 5862:21:28, 18 November 2010 (UTC) 5835:01:18, 19 November 2010 (UTC) 5821:18:27, 18 November 2010 (UTC) 5778:16:50, 18 November 2010 (UTC) 5763:16:48, 18 November 2010 (UTC) 5737:16:08, 18 November 2010 (UTC) 5723:13:41, 18 November 2010 (UTC) 5500:Number of Non-vandalism Edits 5225:03:34, 29 November 2010 (UTC) 5210:03:04, 29 November 2010 (UTC) 5187:02:30, 29 November 2010 (UTC) 5164:01:26, 29 November 2010 (UTC) 5137:01:04, 29 November 2010 (UTC) 5096:00:31, 29 November 2010 (UTC) 5081:00:08, 29 November 2010 (UTC) 5059:23:58, 28 November 2010 (UTC) 5008:21:16, 28 November 2010 (UTC) 4941:20:01, 28 November 2010 (UTC) 4908:04:46, 23 November 2010 (UTC) 4893:21:34, 22 November 2010 (UTC) 4879:20:40, 22 November 2010 (UTC) 4863:20:37, 22 November 2010 (UTC) 4836:19:42, 22 November 2010 (UTC) 4823:19:08, 22 November 2010 (UTC) 4808:18:16, 22 November 2010 (UTC) 4777:20:17, 21 November 2010 (UTC) 4760:01:25, 22 November 2010 (UTC) 4740:01:19, 22 November 2010 (UTC) 4720:16:57, 20 November 2010 (UTC) 4704:16:46, 20 November 2010 (UTC) 4679:16:44, 20 November 2010 (UTC) 4645:20:49, 20 November 2010 (UTC) 4629:on these reported statistics. 4611:17:08, 20 November 2010 (UTC) 4595:08:19, 20 November 2010 (UTC) 4575:04:49, 19 November 2010 (UTC) 4554:16:02, 18 November 2010 (UTC) 4520:04:40, 18 November 2010 (UTC) 4500:23:20, 16 November 2010 (UTC) 4412:Speedy deletion tag handling. 4376:14:50, 15 November 2010 (UTC) 4352:02:12, 22 November 2010 (UTC) 4334:16:14, 15 November 2010 (UTC) 4324:Yes, that helps. Thank you. 4320:14:35, 15 November 2010 (UTC) 4281:11:10, 15 November 2010 (UTC) 4241:07:43, 14 November 2010 (UTC) 4226:05:41, 14 November 2010 (UTC) 4203:21:00, 21 November 2010 (UTC) 4186:16:32, 13 November 2010 (UTC) 4152:22:02, 12 November 2010 (UTC) 4089:16:21, 13 November 2010 (UTC) 4056:00:13, 13 November 2010 (UTC) 4040:22:57, 12 November 2010 (UTC) 4024:20:10, 12 November 2010 (UTC) 3997:15:19, 12 November 2010 (UTC) 3933:07:44, 12 November 2010 (UTC) 3914:03:47, 12 November 2010 (UTC) 3886:16:05, 11 November 2010 (UTC) 3870:15:54, 11 November 2010 (UTC) 3660:09:56, 11 November 2010 (UTC) 3618:02:58, 10 November 2010 (UTC) 3577:02:42, 10 November 2010 (UTC) 3462:16:05, 13 November 2010 (UTC) 3447:08:46, 13 November 2010 (UTC) 3420:13:23, 10 November 2010 (UTC) 3404:13:19, 10 November 2010 (UTC) 3386:08:52, 10 November 2010 (UTC) 2741:16:36, 16 November 2010 (UTC) 2718:07:41, 14 November 2010 (UTC) 2703:06:20, 14 November 2010 (UTC) 2693:05:22, 14 November 2010 (UTC) 2670:05:09, 14 November 2010 (UTC) 2647:16:14, 11 November 2010 (UTC) 2633:16:11, 11 November 2010 (UTC) 2624:16:09, 11 November 2010 (UTC) 2599:03:32, 11 November 2010 (UTC) 2584:03:22, 11 November 2010 (UTC) 1641:04:03, 15 November 2010 (UTC) 1603:17:04, 15 November 2010 (UTC) 1581:22:55, 15 November 2010 (UTC) 1567:22:31, 15 November 2010 (UTC) 1543:22:22, 15 November 2010 (UTC) 1488:04:00, 22 November 2010 (UTC) 764:A couple comments/questions: 6532:02:37, 3 December 2010 (UTC) 6516:02:24, 3 December 2010 (UTC) 5673:04:36, 2 December 2010 (UTC) 5428:04:33, 2 December 2010 (UTC) 5380:10:53, 1 December 2010 (UTC) 5353:09:48, 1 December 2010 (UTC) 5291:"accuracy" after the change. 5271:04:15, 1 December 2010 (UTC) 4618:Just a datapoint among many. 3973:20:33, 4 November 2010 (UTC) 3950:20:22, 4 November 2010 (UTC) 3846:18:49, 9 November 2010 (UTC) 3830:18:31, 9 November 2010 (UTC) 3800:I'm not going to argue that 3784:12:21, 9 November 2010 (UTC) 3743:12:13, 9 November 2010 (UTC) 3733:12:06, 9 November 2010 (UTC) 3718:12:01, 9 November 2010 (UTC) 3698:11:57, 9 November 2010 (UTC) 3558:21:19, 9 November 2010 (UTC) 3548:13:11, 9 November 2010 (UTC) 3529:11:54, 9 November 2010 (UTC) 3504:11:38, 9 November 2010 (UTC) 3488:10:25, 9 November 2010 (UTC) 3366:11:20, 9 November 2010 (UTC) 3351:05:36, 9 November 2010 (UTC) 3322:14:45, 8 November 2010 (UTC) 3308:12:25, 8 November 2010 (UTC) 3293:09:08, 8 November 2010 (UTC) 3277:09:06, 8 November 2010 (UTC) 3259:12:21, 8 November 2010 (UTC) 3244:03:55, 8 November 2010 (UTC) 3220:06:51, 5 November 2010 (UTC) 3204:06:40, 5 November 2010 (UTC) 3169:16:46, 5 November 2010 (UTC) 3154:15:09, 5 November 2010 (UTC) 3140:00:25, 5 November 2010 (UTC) 3125:22:49, 4 November 2010 (UTC) 3099:22:40, 4 November 2010 (UTC) 3064:12:35, 6 November 2010 (UTC) 3041:01:43, 6 November 2010 (UTC) 3026:01:32, 6 November 2010 (UTC) 3001:19:40, 5 November 2010 (UTC) 2992:18:48, 5 November 2010 (UTC) 2976:17:17, 4 November 2010 (UTC) 2950:22:07, 3 November 2010 (UTC) 2921:00:35, 6 November 2010 (UTC) 2906:18:37, 5 November 2010 (UTC) 2888:20:30, 3 November 2010 (UTC) 2879:20:16, 3 November 2010 (UTC) 2857:19:00, 3 November 2010 (UTC) 2843:17:33, 3 November 2010 (UTC) 2830:16:33, 3 November 2010 (UTC) 2814:13:25, 3 November 2010 (UTC) 2790:16:31, 3 November 2010 (UTC) 2775:10:09, 3 November 2010 (UTC) 2561:15:02, 9 November 2010 (UTC) 2546:14:57, 9 November 2010 (UTC) 2525:20:11, 6 November 2010 (UTC) 2505:23:06, 6 November 2010 (UTC) 2490:19:59, 6 November 2010 (UTC) 2470:23:01, 6 November 2010 (UTC) 2456:19:21, 6 November 2010 (UTC) 2436:23:00, 6 November 2010 (UTC) 2421:11:22, 6 November 2010 (UTC) 2401:07:09, 6 November 2010 (UTC) 2386:06:51, 6 November 2010 (UTC) 2360:07:07, 6 November 2010 (UTC) 2346:05:37, 6 November 2010 (UTC) 2322:00:26, 6 November 2010 (UTC) 2307:20:53, 5 November 2010 (UTC) 2281:16:38, 5 November 2010 (UTC) 2266:15:05, 5 November 2010 (UTC) 2246:16:33, 5 November 2010 (UTC) 2231:10:24, 5 November 2010 (UTC) 2208:20:33, 4 November 2010 (UTC) 2193:20:22, 4 November 2010 (UTC) 2176:14:17, 4 November 2010 (UTC) 2162:11:36, 4 November 2010 (UTC) 2143:19:39, 3 November 2010 (UTC) 2128:17:00, 3 November 2010 (UTC) 2111:15:56, 3 November 2010 (UTC) 2097:14:23, 3 November 2010 (UTC) 2060:17:47, 3 November 2010 (UTC) 2029:17:40, 3 November 2010 (UTC) 2011:16:55, 3 November 2010 (UTC) 1979:16:50, 3 November 2010 (UTC) 1965:16:49, 3 November 2010 (UTC) 1951:16:35, 3 November 2010 (UTC) 1933:15:56, 3 November 2010 (UTC) 1918:11:42, 3 November 2010 (UTC) 1894:11:41, 3 November 2010 (UTC) 1860:05:35, 3 November 2010 (UTC) 1804:15:56, 3 November 2010 (UTC) 1789:05:07, 3 November 2010 (UTC) 1766:03:52, 3 November 2010 (UTC) 1751:03:35, 3 November 2010 (UTC) 1728:17:46, 3 November 2010 (UTC) 1714:15:10, 3 November 2010 (UTC) 1693:03:52, 3 November 2010 (UTC) 1678:03:28, 3 November 2010 (UTC) 1621:03:28, 3 November 2010 (UTC) 1457:07:15, 4 November 2010 (UTC) 1442:06:56, 4 November 2010 (UTC) 1428:06:31, 4 November 2010 (UTC) 1390:01:23, 3 November 2010 (UTC) 1370:01:12, 3 November 2010 (UTC) 1352:00:21, 3 November 2010 (UTC) 1338:00:18, 3 November 2010 (UTC) 1319:00:04, 3 November 2010 (UTC) 1289:00:02, 3 November 2010 (UTC) 1274:23:54, 2 November 2010 (UTC) 1263:redirect and redirect =: --> 1217:22:02, 2 November 2010 (UTC) 1196:22:13, 2 November 2010 (UTC) 1182:21:58, 2 November 2010 (UTC) 1167:21:35, 2 November 2010 (UTC) 1149:20:38, 2 November 2010 (UTC) 1095:19:49, 2 November 2010 (UTC) 1079:19:45, 2 November 2010 (UTC) 1056:19:46, 2 November 2010 (UTC) 1039:19:43, 2 November 2010 (UTC) 1024:19:37, 2 November 2010 (UTC) 1002:19:30, 2 November 2010 (UTC) 949:04:37, 2 November 2010 (UTC) 913:21:52, 30 October 2010 (UTC) 880:20:41, 30 October 2010 (UTC) 857:20:11, 30 October 2010 (UTC) 842:20:08, 30 October 2010 (UTC) 826:20:20, 30 October 2010 (UTC) 795:19:34, 30 October 2010 (UTC) 752:12:04, 30 October 2010 (UTC) 737:11:34, 30 October 2010 (UTC) 715:08:29, 28 October 2010 (UTC) 702:05:55, 27 October 2010 (UTC) 687:05:34, 27 October 2010 (UTC) 673:20:40, 26 October 2010 (UTC) 647:19:33, 26 October 2010 (UTC) 622:19:13, 26 October 2010 (UTC) 607:18:59, 26 October 2010 (UTC) 592:06:35, 26 October 2010 (UTC) 582:05:12, 26 October 2010 (UTC) 567:04:37, 26 October 2010 (UTC) 551:03:49, 26 October 2010 (UTC) 519:03:31, 26 October 2010 (UTC) 428:22:46, 25 October 2010 (UTC) 414:22:22, 25 October 2010 (UTC) 400:22:02, 25 October 2010 (UTC) 380:21:03, 25 October 2010 (UTC) 366:19:17, 25 October 2010 (UTC) 351:18:23, 25 October 2010 (UTC) 321:20:22, 4 November 2010 (UTC) 303:06:55, 25 October 2010 (UTC) 287:06:24, 25 October 2010 (UTC) 7: 5438:Major Events During Trial 2 5360:FAQ on CBNG False Positives 4442:Things that can be improved 2370:Du, du liegst mir im Herzen 895:. As you can see from the 10: 6584: 4782:New headers each warning?? 3435:Category:Chemical elements 3433:when they're not in, say, 1326:Greatest Hits... So Far!!! 1306:Greatest Hits... So Far!!! 1302:Greatest Hits... So Far!!! 1231:Ideologia (Cazuza's album) 539:Knowledge:Huggle/Whitelist 502:(while content removal on 6427:had corrected errors and 5236:User talk:ClueBot Commons 5177:is clearly not possible. 5102:by the bot operators only 4736:Penny for your thoughts? 4271:the correct action here? 3470:Riding an old hobby-horse 1416:thin layer chromatography 1298:The Greatest Hits, So Far 6540:Please do not modify it. 5678:False Positive Reporting 5307:effort to classify them. 5172:BRFAs require consensus 4580:False positive exerience 4223:Choyoołʼįįhí:Seb az86556 1840:05:13, 3 November 2010 36:Please do not modify it. 5513:Important Documentation 5026:, as there would be in 4455:End-of-trial statistics 4447:Better markup handling. 3226:possible false positive 2678:), and found two more: 1255:23:45, 2 November 2010 1136:20:05, 2 November 2010 989:19:26, 2 November 2010 901:list of false positives 708:User:ClueBot NG/Dry Run 500:Category:Living persons 6568:Inactive project pages 3824:Questions or Comments? 3356:borderline vandalism. 3181:Helping to classify .. 2515:Again, not vandalism. 1465:False positive reports 1101:I placed a speedy tag 233:Already has a bot flag 154:, and Naomi Amethyst ( 146:Christopher Breneman ( 5915:I don't believe that 5788:Bot undid an edit on 4585:blogs that act up). 4415:Imported opt-in list. 4315:Hope that helps. -- 2954:Thank you to all the 2834:That page was on the 2101:This is being fixed. 2045:was working, but not 1496:article was reverted. 1205:The Secret Peacemaker 22:Requests for approval 6423:Previous version by 6016:? Melodia Chaconne ? 5628:note that this is a 4421:Reverting own edits. 3982:Another Dataset Plea 3910:What did I do wrong? 3678:"Expand the dataset" 3230:I may be wrong, but 265:Pre-trial Discussion 260:Pre-trial Discussion 179:Programming language 5693:Old False Positives 5554:Support for the Bot 5532:false positive info 5464:False Positive Rate 4918:false positive rate 1772:every edit examined 220:Exclusion compliant 4530:Trial 2 discussion 4525:Trial 2 discussion 4477:it should change. 4406:Redirect handling. 2867:User:ClueBot/Optin 1589:This looks wrong. 959:Trial 1 discussion 954:Trial 1 discussion 721:Christopher Boykin 481:The blanking adds 196:Function overview: 6514: 6477: 6476: 6440: 6439: 6253: 6151: 4702: 4677: 3977: 3959: 3956: 3953: 3828: 3716: 3527: 2009: 1904: 1839: 1825:comment added by 1652:Bed Intruder Song 1609:the link provided 1254: 1240:comment added by 1229:and to the album 1135: 1121:comment added by 988: 974:comment added by 947: 732: 340:other approaches? 285: 245:Function details: 6575: 6542: 6513: 6510: 6505: 6498: 6493: 6492: 6473: 6456: 6449: 6420: 6391: 6376: 6329: 6249: 6235: 6144: 6140: 6105: 6036: 6007: 5912: 5845: 5747: 5697: 5696: 5636:improved, etc). 5426: 5424: 5419: 5409: 5408: 4804: 4799: 4794: 4737: 4731: 4699: 4693: 4688: 4674: 4668: 4663: 4508: 4386: 4385: 4117: 4112:Old revision of 4053: 4012: 4006: 3961: 3958: 3955: 3938: 3931: 3929: 3924: 3912: 3907: 3905: 3898: 3825: 3820: 3818: 3815: 3812: 3809: 3796:Multiple reverts 3709: 3705: 3535:exactly the same 3520: 3514: 3509: 3121: 3116: 3109: 3095: 3090: 3083: 3015: 3009: 2931:Review Interface 2531: 2530: 2301: 2093: 2087: 2082: 2078:by Uncle Milty. 2054: 2048: 2044: 2038: 2002: 1998: 1992: 1986: 1898: 1891: 1885: 1880: 1838: 1819: 1253: 1234: 1134: 1115: 1110: 1104: 987: 968: 940: 934: 929: 923: 922: 854: 839: 793: 731: 728: 727: 683: 490: 484: 477: 471: 334:provided by the 284: 282: 50: 38: 6583: 6582: 6578: 6577: 6576: 6574: 6573: 6572: 6553: 6552: 6551: 6538: 6508: 6503: 6500: 6490: 6487: 6479: 6478: 6467: 6446: 6445:False Positives 6442: 6441: 6416: 6387: 6372: 6325: 6245:Rambo's Revenge 6231: 6142: 6101: 6032: 6003: 5921:5 albert square 5908: 5841: 5743: 5694: 5680: 5540:post-processing 5526:, particularly 5435: 5433:Trial 2 Summary 5422: 5417: 5414: 5411:Trial complete. 5406: 5403: 5402: 5368:CBNG Algorithms 4920: 4850:edit by the bot 4846: 4844:Race condition? 4802: 4797: 4792: 4784: 4735: 4727: 4697: 4691: 4672: 4666: 4656: 4582: 4562: 4537: 4531: 4527: 4409:Quote handling. 4395: 4388:Trial complete. 4383: 4380: 4379: 4359: 4249: 4217:what triggered 4215: 4110: 4101: 4068:classification. 4047: 4010: 4004: 3984: 3927: 3922: 3920: 3908: 3901: 3894: 3892: 3854: 3823: 3816: 3813: 3810: 3807: 3798: 3707: 3680: 3518: 3512: 3472: 3228: 3183: 3119: 3112: 3107: 3093: 3086: 3081: 3072: 3013: 3007: 2933: 2798: 2796:False positives 2760: 2295: 2154:174.109.197.174 2091: 2085: 2080: 2052: 2046: 2042: 2036: 2000: 1990: 1984: 1889: 1881: 1876: 1820: 1595:131.202.131.250 1467: 1377:User:Cluebot_NG 1235: 1116: 1108: 1102: 969: 960: 956: 938: 932: 920: 917: 916: 848: 833: 781: 729: 725: 681: 534:User:Tide rolls 524:I also noticed 488: 482: 475: 469: 454:WP:NOT#CENSORED 336:CLEF 2010 LABs? 280: 266: 262: 257: 207:Edit period(s): 141: 80: 70: 60: 59:False Positives 34: 26: 25: 24: 12: 11: 5: 6581: 6571: 6570: 6565: 6550: 6549: 6534: 6486: 6483: 6481: 6475: 6474: 6466: 6457: 6447: 6444: 6443: 6438: 6437: 6435: 6432: 6421: 6413: 6412: 6409: 6406: 6400: 6399: 6395: 6392: 6384: 6383: 6380: 6377: 6369: 6368: 6365: 6351: 6348: 6347: 6344: 6330: 6322: 6321: 6319: 6313: 6310: 6309: 6307: 6304: 6298: 6297: 6282: 6276: 6273: 6272: 6258: 6236: 6228: 6227: 6226: 6225: 6224: 6223: 6183:Philip Trueman 6179: 6163: 6158: 6157: 6156: 6155: 6123: 6113:Philip Trueman 6106: 6098: 6097: 6082: 6064: 6058: 6057: 6055: 6037: 6029: 6028: 6026: 6008: 6000: 5999: 5997: 5979: 5976: 5975: 5973: 5958: 5955: 5954: 5952: 5937: 5934: 5933: 5931: 5913: 5905: 5904: 5902: 5899: 5896: 5895: 5893: 5890: 5887: 5886: 5884: 5870: 5867: 5866: 5864: 5854:Confession0791 5846: 5838: 5837: 5823: 5786: 5781: 5780: 5765: 5748: 5740: 5739: 5725: 5711: 5708: 5707: 5704: 5701: 5695: 5692: 5691: 5689: 5679: 5676: 5657: 5656: 5653: 5646: 5645: 5625: 5624: 5617: 5616: 5611: 5606: 5601: 5596: 5591: 5586: 5581: 5576: 5571: 5566: 5556: 5555: 5551: 5550: 5543: 5515: 5514: 5510: 5509: 5503: 5485: 5484: 5483:Clarifications 5480: 5479: 5473: 5467: 5457: 5456: 5452: 5451: 5448: 5445: 5440: 5439: 5434: 5431: 5401: 5400: 5399: 5398: 5397: 5396: 5395: 5394: 5393: 5392: 5391: 5390: 5389: 5388: 5387: 5386: 5385: 5384: 5383: 5382: 5355: 5332: 5328: 5324: 5316: 5312: 5308: 5304: 5300: 5296: 5292: 5288: 5281: 5277: 5258: 5197: 5193: 5170: 5151: 5147: 5143: 5118: 5114: 5083: 5068: 5064: 5039: 5014: 4995: 4994: 4993: 4990: 4986: 4983: 4980: 4977: 4974: 4971: 4968: 4961: 4958: 4954: 4951: 4919: 4916: 4915: 4914: 4913: 4912: 4911: 4910: 4845: 4842: 4841: 4840: 4839: 4838: 4783: 4780: 4767: 4766: 4765: 4764: 4763: 4762: 4655: 4652: 4648: 4647: 4631: 4630: 4614: 4613: 4581: 4578: 4567:Philip Trueman 4561: 4558: 4557: 4556: 4536: 4533: 4532: 4529: 4528: 4526: 4523: 4487: 4486: 4470: 4469: 4465: 4464: 4461: 4457: 4456: 4452: 4451: 4448: 4444: 4443: 4439: 4438: 4435: 4432: 4428: 4427: 4423: 4422: 4419: 4416: 4413: 4410: 4407: 4403: 4402: 4394: 4391: 4358: 4355: 4339: 4338: 4337: 4336: 4326:Philip Trueman 4313: 4312: 4311: 4308: 4305: 4302: 4298: 4295: 4292: 4289: 4273:Philip Trueman 4248: 4245: 4244: 4243: 4214: 4211: 4210: 4209: 4208: 4207: 4206: 4205: 4175: 4174: 4173: 4170: 4166: 4162: 4155: 4154: 4140: 4136: 4133: 4130: 4100: 4097: 4096: 4095: 4094: 4093: 4092: 4091: 4078: 4077: 4076: 4073: 4069: 4065: 4030: 4026: 4016:Philip Trueman 3983: 3980: 3979: 3978: 3936: 3935: 3889: 3888: 3853: 3850: 3849: 3848: 3797: 3794: 3793: 3792: 3791: 3790: 3789: 3788: 3787: 3786: 3771: 3767: 3764: 3763: 3762: 3758: 3755: 3752: 3679: 3676: 3675: 3674: 3673: 3672: 3671: 3670: 3669: 3668: 3667: 3666: 3665: 3664: 3663: 3662: 3652:Philip Trueman 3631: 3630: 3629: 3628: 3627: 3626: 3625: 3624: 3623: 3622: 3621: 3620: 3594: 3593: 3592: 3591: 3590: 3589: 3588: 3587: 3586: 3585: 3584: 3583: 3569:Philip Trueman 3564: 3540:Philip Trueman 3480:Philip Trueman 3471: 3468: 3467: 3466: 3465: 3464: 3439:Philip Trueman 3423: 3422: 3407: 3406: 3396:Philip Trueman 3389: 3388: 3378:Philip Trueman 3371: 3370: 3369: 3368: 3343:Philip Trueman 3338: 3337: 3329: 3328: 3327: 3326: 3325: 3324: 3314:Philip Trueman 3280: 3279: 3269:Philip Trueman 3262: 3261: 3227: 3224: 3223: 3222: 3196:Philip Trueman 3182: 3179: 3178: 3177: 3176: 3175: 3174: 3173: 3172: 3171: 3071: 3068: 3067: 3066: 3056:Philip Trueman 3050: 3049: 3048: 3047: 3046: 3045: 3044: 3043: 3018:Philip Trueman 2984:Philip Trueman 2932: 2929: 2928: 2927: 2926: 2925: 2924: 2923: 2898:Philip Trueman 2894: 2893: 2892: 2891: 2890: 2859: 2797: 2794: 2793: 2792: 2759: 2756: 2755: 2754: 2744: 2743: 2725: 2724: 2723: 2722: 2721: 2720: 2672: 2655: 2654: 2653: 2652: 2651: 2650: 2649: 2606: 2605: 2604: 2603: 2602: 2601: 2576:Philip Trueman 2566: 2565: 2564: 2563: 2535: 2534: 2533: 2532: 2510: 2509: 2508: 2507: 2475: 2474: 2473: 2472: 2441: 2440: 2439: 2438: 2413:Philip Trueman 2406: 2405: 2404: 2403: 2365: 2364: 2363: 2362: 2327: 2326: 2325: 2324: 2286: 2285: 2284: 2283: 2258:Philip Trueman 2251: 2250: 2249: 2248: 2219:Franco Selleri 2211: 2210: 2181: 2180: 2179: 2178: 2116: 2115: 2114: 2113: 2072: 2071: 2070: 2069: 2068: 2067: 2066: 2065: 2064: 2063: 2062: 2033: 2032: 2031: 1981: 1906:here's another 1868: 1867: 1866: 1865: 1813: 1812: 1811: 1810: 1809: 1808: 1807: 1806: 1735: 1734: 1733: 1732: 1731: 1730: 1698: 1697: 1696: 1695: 1656:Allyisaunicorn 1644: 1643: 1629: 1624: 1623: 1605: 1586: 1585: 1584: 1583: 1546: 1545: 1527: 1523: 1512: 1505: 1498: 1497: 1490: 1466: 1463: 1462: 1461: 1460: 1459: 1398:80.192.184.101 1395: 1394: 1393: 1392: 1357: 1356: 1355: 1354: 1294: 1293: 1292: 1291: 1201: 1200: 1199: 1198: 1154: 1153: 1152: 1151: 1098: 1097: 1067: 1066: 1065: 1064: 1063: 1062: 1061: 1060: 1059: 1058: 961: 958: 957: 955: 952: 889: 888: 887: 886: 885: 884: 883: 882: 867: 863: 828: 813: 812: 811: 807: 803: 778: 777: 776: 772: 768: 762: 761: 760: 759: 758: 757: 756: 755: 754: 660: 659: 658: 657: 656: 655: 654: 653: 652: 651: 650: 649: 522: 521: 496: 492: 479: 441: 440: 439: 438: 437: 436: 435: 434: 433: 432: 431: 430: 326: 325: 324: 323: 305: 267: 264: 263: 261: 258: 256: 253: 140: 139: 133: 128: 123: 118: 113: 108: 103: 98: 93: 91:Approved BRFAs 88: 81: 79: 74: 72: 68: 58: 56: 55: 29: 18:Knowledge:Bots 15: 9: 6: 4: 3: 2: 6580: 6569: 6566: 6564: 6561: 6560: 6558: 6548: 6546: 6541: 6535: 6533: 6529: 6525: 6520: 6519: 6518: 6517: 6512: 6511: 6506: 6496: 6482: 6471: 6465: 6463: 6458: 6455: 6451: 6450: 6436: 6433: 6430: 6426: 6425:70.26.181.136 6422: 6419: 6415: 6414: 6410: 6407: 6405: 6402: 6401: 6396: 6393: 6390: 6386: 6385: 6381: 6378: 6375: 6371: 6370: 6366: 6364: 6360: 6356: 6355:173.59.219.49 6352: 6350: 6349: 6345: 6343: 6339: 6335: 6331: 6328: 6324: 6323: 6320: 6318: 6314: 6312: 6311: 6308: 6305: 6303: 6302:Shawn Johnson 6300: 6299: 6296: 6292: 6288: 6283: 6281: 6277: 6275: 6274: 6271: 6267: 6263: 6259: 6257: 6254: 6252: 6247: 6246: 6241: 6237: 6234: 6230: 6229: 6222: 6218: 6214: 6209: 6208: 6207: 6203: 6199: 6194: 6193: 6192: 6188: 6184: 6180: 6178: 6174: 6170: 6169:130.233.79.47 6166: 6164: 6162: 6160: 6159: 6154: 6150: 6146: 6138: 6137: 6136: 6132: 6128: 6124: 6122: 6118: 6114: 6110: 6107: 6104: 6100: 6099: 6096: 6092: 6088: 6083: 6081: 6077: 6073: 6069: 6065: 6063: 6060: 6059: 6056: 6054: 6050: 6046: 6042: 6038: 6035: 6031: 6030: 6027: 6025: 6021: 6017: 6013: 6010:I wonder why 6009: 6006: 6002: 6001: 5998: 5996: 5992: 5988: 5984: 5980: 5978: 5977: 5974: 5972: 5968: 5964: 5959: 5957: 5956: 5953: 5950: 5946: 5942: 5941:83.70.226.255 5938: 5936: 5935: 5932: 5930: 5926: 5922: 5918: 5914: 5911: 5907: 5906: 5903: 5900: 5898: 5897: 5894: 5891: 5889: 5888: 5885: 5883: 5879: 5875: 5871: 5869: 5868: 5865: 5863: 5860: 5859: 5856: 5855: 5850: 5847: 5844: 5840: 5839: 5836: 5832: 5828: 5824: 5822: 5818: 5814: 5810: 5806: 5802: 5798: 5794: 5790: 5787: 5785: 5783: 5782: 5779: 5775: 5771: 5766: 5764: 5760: 5756: 5752: 5749: 5746: 5742: 5741: 5738: 5734: 5730: 5726: 5724: 5720: 5716: 5712: 5710: 5709: 5705: 5702: 5699: 5698: 5690: 5687: 5685: 5675: 5674: 5670: 5666: 5661: 5654: 5651: 5650: 5649: 5643: 5642: 5641: 5637: 5633: 5631: 5622: 5621: 5620: 5615: 5612: 5610: 5607: 5605: 5602: 5600: 5597: 5595: 5592: 5590: 5587: 5585: 5582: 5580: 5577: 5575: 5572: 5570: 5567: 5565: 5562: 5561: 5560: 5553: 5552: 5548: 5544: 5541: 5537: 5533: 5529: 5525: 5521: 5520: 5519: 5512: 5511: 5507: 5504: 5501: 5497: 5493: 5490: 5489: 5488: 5482: 5481: 5477: 5474: 5471: 5468: 5465: 5462: 5461: 5460: 5455:Controversies 5454: 5453: 5449: 5446: 5442: 5441: 5437: 5436: 5430: 5429: 5425: 5420: 5412: 5381: 5377: 5373: 5369: 5365: 5361: 5356: 5354: 5350: 5346: 5342: 5337: 5333: 5329: 5325: 5321: 5317: 5313: 5309: 5305: 5301: 5297: 5293: 5289: 5286: 5282: 5278: 5274: 5273: 5272: 5268: 5264: 5263:Peter Karlsen 5259: 5255: 5252: 5248: 5245: 5241: 5237: 5232: 5228: 5227: 5226: 5222: 5218: 5213: 5212: 5211: 5207: 5203: 5198: 5194: 5190: 5189: 5188: 5184: 5180: 5179:Peter Karlsen 5175: 5171: 5167: 5166: 5165: 5161: 5157: 5152: 5148: 5144: 5140: 5139: 5138: 5134: 5130: 5129:Peter Karlsen 5126: 5122: 5119: 5115: 5111: 5107: 5103: 5099: 5098: 5097: 5093: 5089: 5084: 5082: 5078: 5074: 5069: 5065: 5062: 5061: 5060: 5056: 5052: 5051:Peter Karlsen 5048: 5044: 5040: 5037: 5033: 5029: 5025: 5020: 5015: 5011: 5010: 5009: 5005: 5001: 4996: 4991: 4987: 4984: 4981: 4978: 4975: 4972: 4969: 4966: 4962: 4959: 4955: 4952: 4948: 4947: 4945: 4944: 4943: 4942: 4938: 4934: 4930: 4924: 4909: 4905: 4901: 4900:Peter Karlsen 4896: 4895: 4894: 4890: 4886: 4882: 4881: 4880: 4876: 4872: 4867: 4866: 4865: 4864: 4860: 4856: 4851: 4837: 4834: 4830: 4826: 4825: 4824: 4820: 4816: 4812: 4811: 4810: 4809: 4806: 4805: 4800: 4795: 4788: 4779: 4778: 4775: 4771: 4761: 4757: 4753: 4748: 4743: 4742: 4741: 4738: 4732: 4730: 4723: 4722: 4721: 4717: 4713: 4708: 4707: 4706: 4705: 4701: 4700: 4694: 4692:Access Denied 4686: 4683:I also found 4681: 4680: 4676: 4675: 4669: 4667:Access Denied 4661: 4651: 4646: 4642: 4638: 4633: 4632: 4627: 4626: 4625: 4621: 4619: 4612: 4608: 4604: 4599: 4598: 4597: 4596: 4592: 4588: 4577: 4576: 4572: 4568: 4565: 4555: 4551: 4547: 4542: 4541: 4540: 4522: 4521: 4518: 4515: 4511: 4507: 4502: 4501: 4497: 4493: 4484: 4483: 4482: 4478: 4474: 4467: 4466: 4462: 4459: 4458: 4454: 4453: 4449: 4446: 4445: 4441: 4440: 4436: 4433: 4430: 4429: 4425: 4424: 4420: 4417: 4414: 4411: 4408: 4405: 4404: 4400: 4399: 4398: 4393:Trial Summary 4390: 4389: 4378: 4377: 4373: 4369: 4363: 4357:Status Update 4354: 4353: 4349: 4345: 4335: 4331: 4327: 4323: 4322: 4321: 4318: 4314: 4309: 4306: 4303: 4299: 4296: 4293: 4290: 4287: 4286: 4285: 4284: 4283: 4282: 4278: 4274: 4268: 4264: 4260: 4256: 4252: 4242: 4238: 4234: 4230: 4229: 4228: 4227: 4224: 4220: 4204: 4200: 4196: 4191: 4190: 4189: 4188: 4187: 4183: 4179: 4176: 4171: 4167: 4163: 4160: 4159: 4157: 4156: 4153: 4150: 4149: 4146: 4141: 4137: 4134: 4131: 4128: 4127: 4126: 4124: 4119: 4116: 4115: 4107: 4090: 4086: 4082: 4079: 4074: 4070: 4066: 4062: 4061: 4059: 4058: 4057: 4054: 4052: 4051: 4043: 4042: 4041: 4038: 4036: 4031: 4027: 4025: 4021: 4017: 4009: 4001: 4000: 3999: 3998: 3994: 3990: 3976: 3974: 3970: 3966: 3960: 3957: 3954: 3952: 3951: 3947: 3943: 3934: 3930: 3925: 3918: 3917: 3916: 3915: 3911: 3906: 3904: 3899: 3897: 3896:Allmightyduck 3887: 3883: 3879: 3874: 3873: 3872: 3871: 3867: 3863: 3859: 3847: 3843: 3839: 3834: 3833: 3832: 3831: 3826: 3819: 3803: 3802:these reverts 3785: 3781: 3777: 3772: 3768: 3765: 3759: 3756: 3753: 3750: 3749: 3746: 3745: 3744: 3741: 3736: 3735: 3734: 3730: 3726: 3721: 3720: 3719: 3715: 3711: 3702: 3701: 3700: 3699: 3695: 3691: 3686: 3661: 3657: 3653: 3648: 3645: 3644: 3643: 3642: 3641: 3640: 3639: 3638: 3637: 3636: 3635: 3634: 3633: 3632: 3619: 3615: 3611: 3606: 3605: 3604: 3603: 3602: 3601: 3600: 3599: 3598: 3597: 3596: 3595: 3580: 3579: 3578: 3574: 3570: 3565: 3561: 3560: 3559: 3556: 3551: 3550: 3549: 3545: 3541: 3536: 3532: 3531: 3530: 3525: 3521: 3515: 3507: 3506: 3505: 3501: 3497: 3492: 3491: 3490: 3489: 3485: 3481: 3476: 3463: 3459: 3455: 3450: 3449: 3448: 3444: 3440: 3436: 3432: 3427: 3425: 3424: 3421: 3417: 3413: 3409: 3408: 3405: 3401: 3397: 3393: 3391: 3390: 3387: 3383: 3379: 3375: 3373: 3372: 3367: 3363: 3359: 3354: 3353: 3352: 3348: 3344: 3340: 3339: 3336: 3333: 3331: 3330: 3323: 3319: 3315: 3311: 3310: 3309: 3305: 3301: 3296: 3295: 3294: 3290: 3286: 3282: 3281: 3278: 3274: 3270: 3266: 3264: 3263: 3260: 3256: 3252: 3248: 3247: 3246: 3245: 3241: 3237: 3233: 3221: 3217: 3213: 3208: 3207: 3206: 3205: 3201: 3197: 3192: 3189: 3187: 3170: 3166: 3162: 3157: 3156: 3155: 3151: 3147: 3143: 3142: 3141: 3137: 3133: 3128: 3127: 3126: 3123: 3122: 3117: 3115: 3110: 3103: 3102: 3101: 3100: 3097: 3096: 3091: 3089: 3084: 3077: 3065: 3061: 3057: 3052: 3051: 3042: 3038: 3034: 3029: 3028: 3027: 3023: 3019: 3012: 3004: 3003: 3002: 2999: 2995: 2994: 2993: 2989: 2985: 2980: 2979: 2978: 2977: 2973: 2969: 2963: 2961: 2957: 2952: 2951: 2947: 2943: 2939: 2922: 2918: 2914: 2909: 2908: 2907: 2903: 2899: 2895: 2889: 2886: 2882: 2881: 2880: 2876: 2872: 2871:Peter Karlsen 2868: 2864: 2860: 2858: 2854: 2850: 2846: 2845: 2844: 2841: 2837: 2833: 2832: 2831: 2827: 2823: 2818: 2817: 2816: 2815: 2811: 2807: 2803: 2791: 2787: 2783: 2779: 2778: 2777: 2776: 2772: 2768: 2764: 2752: 2748: 2746: 2745: 2742: 2738: 2734: 2730: 2727: 2726: 2719: 2715: 2711: 2706: 2705: 2704: 2701: 2696: 2695: 2694: 2690: 2686: 2682: 2680: 2677: 2673: 2671: 2667: 2663: 2658: 2656: 2648: 2644: 2640: 2636: 2635: 2634: 2631: 2627: 2626: 2625: 2621: 2617: 2613: 2612: 2610: 2608: 2607: 2600: 2596: 2592: 2587: 2586: 2585: 2581: 2577: 2573: 2572: 2570: 2568: 2567: 2562: 2558: 2554: 2549: 2548: 2547: 2544: 2539: 2537: 2536: 2528: 2527: 2526: 2523: 2522: 2519: 2514: 2512: 2511: 2506: 2502: 2498: 2493: 2492: 2491: 2488: 2487: 2484: 2479: 2477: 2476: 2471: 2467: 2463: 2459: 2458: 2457: 2454: 2453: 2450: 2445: 2443: 2442: 2437: 2433: 2429: 2424: 2423: 2422: 2418: 2414: 2410: 2408: 2407: 2402: 2398: 2394: 2389: 2388: 2387: 2383: 2379: 2378:71.38.118.252 2375: 2371: 2367: 2366: 2361: 2357: 2353: 2349: 2348: 2347: 2344: 2343: 2340: 2335: 2331: 2329: 2328: 2323: 2319: 2315: 2310: 2309: 2308: 2305: 2302: 2300: 2299: 2291: 2288: 2287: 2282: 2278: 2274: 2269: 2268: 2267: 2263: 2259: 2255: 2253: 2252: 2247: 2243: 2239: 2234: 2233: 2232: 2228: 2224: 2220: 2216: 2213: 2212: 2209: 2205: 2201: 2197: 2196: 2195: 2194: 2190: 2186: 2177: 2173: 2169: 2165: 2164: 2163: 2159: 2155: 2151: 2147: 2146: 2145: 2144: 2140: 2136: 2130: 2129: 2125: 2121: 2112: 2108: 2104: 2100: 2099: 2098: 2094: 2088: 2083: 2077: 2073: 2061: 2058: 2051: 2041: 2034: 2030: 2026: 2022: 2018: 2014: 2013: 2012: 2008: 2004: 1996: 1989: 1982: 1980: 1976: 1972: 1968: 1967: 1966: 1962: 1958: 1954: 1953: 1952: 1948: 1944: 1940: 1936: 1935: 1934: 1930: 1926: 1921: 1920: 1919: 1915: 1911: 1907: 1902: 1901:edit conflict 1897: 1896: 1895: 1892: 1886: 1884: 1879: 1873: 1870: 1869: 1863: 1862: 1861: 1857: 1853: 1852:Peter Karlsen 1849: 1846: 1843: 1842: 1841: 1836: 1832: 1828: 1824: 1818: 1805: 1801: 1797: 1792: 1791: 1790: 1786: 1782: 1781:Peter Karlsen 1777: 1773: 1769: 1768: 1767: 1763: 1759: 1754: 1753: 1752: 1748: 1744: 1743:Peter Karlsen 1740: 1737: 1736: 1729: 1725: 1721: 1717: 1716: 1715: 1711: 1707: 1702: 1701: 1700: 1699: 1694: 1690: 1686: 1681: 1680: 1679: 1675: 1671: 1667: 1664: 1661: 1657: 1653: 1649: 1646: 1645: 1642: 1638: 1634: 1630: 1626: 1625: 1622: 1618: 1614: 1610: 1606: 1604: 1600: 1596: 1592: 1588: 1587: 1582: 1578: 1574: 1573:24.72.122.184 1570: 1569: 1568: 1564: 1560: 1559:24.72.122.184 1556: 1552: 1548: 1547: 1544: 1540: 1536: 1535:24.72.122.184 1532: 1528: 1524: 1521: 1517: 1516:70.173.230.88 1513: 1510: 1506: 1503: 1502: 1501: 1495: 1491: 1489: 1485: 1481: 1480:75.67.220.101 1477: 1473: 1469: 1468: 1458: 1454: 1450: 1445: 1444: 1443: 1440: 1436: 1432: 1431: 1430: 1429: 1425: 1421: 1417: 1413: 1409: 1406: 1403: 1399: 1391: 1387: 1383: 1378: 1373: 1372: 1371: 1367: 1363: 1359: 1358: 1353: 1349: 1345: 1341: 1340: 1339: 1335: 1331: 1327: 1323: 1322: 1321: 1320: 1316: 1312: 1307: 1303: 1299: 1290: 1286: 1282: 1277: 1276: 1275: 1271: 1267: 1266:Peter Karlsen 1261: 1258: 1257: 1256: 1251: 1247: 1243: 1242:187.68.100.92 1239: 1232: 1228: 1224: 1219: 1218: 1214: 1210: 1207:for details. 1206: 1197: 1193: 1189: 1185: 1184: 1183: 1179: 1175: 1171: 1170: 1169: 1168: 1164: 1160: 1150: 1146: 1142: 1138: 1137: 1132: 1128: 1124: 1123:81.145.247.25 1120: 1114:Thanks, Jon. 1113: 1107: 1100: 1099: 1096: 1092: 1088: 1083: 1082: 1081: 1080: 1076: 1072: 1071:Carl Sixsmith 1057: 1053: 1049: 1048:Peter Karlsen 1045: 1042: 1041: 1040: 1036: 1032: 1027: 1026: 1025: 1021: 1017: 1016:Peter Karlsen 1013: 1011: 1008: 1005: 1004: 1003: 999: 995: 991: 990: 985: 981: 977: 976:81.145.247.25 973: 966: 963: 962: 951: 950: 945: 941: 935: 926: 915: 914: 910: 906: 902: 898: 894: 881: 877: 873: 868: 864: 860: 859: 858: 855: 853: 852: 845: 844: 843: 840: 838: 837: 829: 827: 823: 819: 814: 808: 804: 800: 799: 798: 797: 796: 792: 791: 789: 784: 779: 773: 771:learn faster? 769: 766: 765: 763: 753: 749: 745: 740: 739: 738: 735: 733: 722: 718: 717: 716: 713: 709: 705: 704: 703: 699: 695: 690: 689: 688: 685: 684: 677: 676: 675: 674: 670: 666: 648: 644: 640: 639:Peter Karlsen 635: 632: 629: 625: 624: 623: 619: 615: 610: 609: 608: 604: 600: 595: 594: 593: 590: 585: 584: 583: 579: 575: 574:Peter Karlsen 570: 569: 568: 564: 560: 555: 554: 553: 552: 548: 544: 543:Peter Karlsen 540: 535: 531: 527: 520: 516: 512: 511:Peter Karlsen 508: 505: 501: 497: 493: 487: 480: 474: 467: 466: 465: 463: 459: 455: 451: 448: 446: 429: 425: 421: 417: 416: 415: 411: 407: 403: 402: 401: 397: 393: 390: 387: 383: 382: 381: 377: 373: 369: 368: 367: 363: 359: 354: 353: 352: 348: 344: 341: 337: 333: 328: 327: 322: 318: 314: 309: 308: 306: 304: 300: 296: 291: 290: 289: 288: 283: 276: 272: 252: 250: 246: 242: 240: 237: 234: 230: 228: 225: 222: 221: 216: 214: 210: 208: 204: 203: 199: 197: 193: 191: 189: 184: 182: 180: 175: 173: 169: 167: 163: 159: 157: 156:NaomiAmethyst 153: 149: 145: 137: 134: 132: 129: 127: 124: 122: 119: 117: 114: 112: 109: 107: 104: 102: 99: 97: 94: 92: 89: 87: 83: 82: 78: 73: 67: 65: 53: 49: 44: 42: 37: 31: 30: 23: 19: 6539: 6536: 6501: 6494: 6488: 6480: 6470:village pump 6459: 6434: 6428: 6334:Followorders 6316: 6279: 6250: 6244: 5987:192.12.88.68 5961:corrected).— 5874:216.96.2.184 5857: 5852: 5715:78.72.250.55 5688: 5681: 5662: 5658: 5647: 5638: 5634: 5629: 5626: 5618: 5557: 5516: 5505: 5499: 5495: 5491: 5486: 5475: 5469: 5463: 5458: 5410: 5404: 5340: 5335: 5319: 5284: 5243: 5173: 5124: 5109: 5105: 5101: 5046: 5018: 4964: 4925: 4921: 4847: 4829:add a header 4790: 4785: 4769:Additional: 4768: 4746: 4728: 4689: 4682: 4664: 4658:For example 4657: 4649: 4622: 4617: 4615: 4587:72.82.33.250 4583: 4563: 4538: 4509: 4503: 4488: 4479: 4475: 4471: 4396: 4387: 4381: 4364: 4360: 4340: 4269: 4265: 4261: 4257: 4253: 4250: 4216: 4147: 4144: 4122: 4120: 4113: 4105: 4102: 4049: 4048: 3985: 3962: 3939: 3937: 3902: 3895: 3890: 3855: 3799: 3684: 3681: 3534: 3473: 3229: 3193: 3190: 3184: 3113: 3105: 3087: 3079: 3073: 2964: 2953: 2938:at this link 2934: 2861:Why did you 2799: 2761: 2520: 2517: 2485: 2482: 2451: 2448: 2341: 2338: 2297: 2296: 2289:This revert 2182: 2131: 2117: 1882: 1877: 1821:— Preceding 1814: 1775: 1771: 1662: 1633:Happybunny95 1631:Same thing. 1554: 1550: 1499: 1404: 1396: 1295: 1236:— Preceding 1220: 1202: 1155: 1117:— Preceding 1106:db-redirnone 1068: 970:— Preceding 924: 918: 890: 850: 849: 835: 834: 787: 786: 679: 661: 523: 506: 457: 442: 274: 270: 268: 244: 243: 238: 235: 232: 231: 226: 223: 218: 217: 212: 211: 209:Continuous. 206: 205: 201: 200: 195: 194: 186: 185: 177: 176: 171: 170: 161: 160: 143: 142: 85: 71: 61: 51: 35: 32: 6072:71.91.99.47 5963:Jerome Kohl 5809:SMS/Texting 5801:SML/Texting 5797:SMS/Texting 5793:SML/Texting 5640:even more. 5569:Awards page 5564:Praise page 5522:The entire 5358:concisely: 5032:edit filter 4950:vandalism". 4729:HJ Mitchell 3858:second edit 1607:I followed 1342:Thank you! 587:* 100. -- 188:Source code 174:Automatic. 162:Time filed: 6557:Categories 6524:Crispy1989 6464:reference. 6462:historical 6287:Crispy1989 6262:Crispy1989 6213:Crispy1989 6198:Crispy1989 6127:Crispy1989 6087:Crispy1989 5827:Crispy1989 5813:Phoenixkin 5770:Crispy1989 5755:ChiZeroOne 5729:Crispy1989 5665:Crispy1989 5372:Crispy1989 5345:Crispy1989 5217:Crispy1989 5202:Crispy1989 5156:Crispy1989 5088:Crispy1989 5073:Crispy1989 5000:Crispy1989 4871:Crispy1989 4815:Crispy1989 4752:Crispy1989 4712:Crispy1989 4698:talk to me 4673:talk to me 4637:Crispy1989 4603:Crispy1989 4546:Crispy1989 4492:Crispy1989 4368:Crispy1989 4344:Crispy1989 4233:Crispy1989 4178:Crispy1989 4081:Crispy1989 4008:Persondata 3989:Crispy1989 3965:Crispy1989 3878:Crispy1989 3838:Crispy1989 3776:Crispy1989 3610:Crispy1989 3496:Crispy1989 3474:This edit 3454:Crispy1989 3412:Crispy1989 3358:Crispy1989 3300:Crispy1989 3251:Crispy1989 3212:Crispy1989 3161:Crispy1989 3132:Crispy1989 3033:Crispy1989 3011:Persondata 2968:Crispy1989 2942:Crispy1989 2913:Crispy1989 2836:optin list 2822:Crispy1989 2782:Crispy1989 2710:Crispy1989 2685:Shubinator 2662:Shubinator 2639:Crispy1989 2616:Crispy1989 2591:Crispy1989 2553:Crispy1989 2497:Crispy1989 2462:Crispy1989 2428:Crispy1989 2393:Crispy1989 2352:Crispy1989 2334:ionization 2314:Crispy1989 2273:Crispy1989 2238:Crispy1989 2200:Crispy1989 2168:Crispy1989 2150:Ajharper18 2135:Crispy1989 2120:Crispy1989 2103:Crispy1989 2021:Crispy1989 1957:Crispy1989 1925:Crispy1989 1796:Crispy1989 1758:Crispy1989 1720:Crispy1989 1685:Crispy1989 1670:Bill Price 1613:Bill Price 1476:Shell game 1470:I made an 1449:Crispy1989 1382:Crispy1989 1330:Crispy1989 1281:Crispy1989 1188:Crispy1989 1159:Crispy1989 1141:Crispy1989 1087:Crispy1989 1031:Crispy1989 994:Crispy1989 905:Crispy1989 872:Crispy1989 818:Crispy1989 744:Crispy1989 694:Crispy1989 665:Crispy1989 614:Crispy1989 559:Crispy1989 420:Crispy1989 392:Crispy1989 358:Crispy1989 295:Crispy1989 255:Discussion 190:available: 148:Crispy1989 131:rights log 121:page moves 77:ClueBot NG 64:do so here 6495:Approved. 6280:Thanks! 6041:this one, 5983:swan song 5536:threshold 5524:user page 5113:concerns. 4933:Stepheng3 4560:Thank you 4139:message.) 4123:necessity 3862:Millahnna 3285:Millahnna 3232:this edit 3076:this edit 2849:Sf5xeplus 2806:Sf5xeplus 2763:this edit 2729:This edit 2223:Crowsnest 2215:This edit 2017:User:Cobi 1776:well over 1648:This edit 1628:accuracy. 1531:this page 1412:test edit 1410:) made a 1259:Reverted 1227:Ideología 1223:Ideologia 1174:Sole Soul 526:this diff 343:Sole Soul 144:Operator: 126:block log 6485:Approval 6429:improved 6045:Cynwolfe 5423:Amethyst 5285:maximums 5150:in size. 5047:probably 5019:reverted 4885:User:Jwy 4855:User:Jwy 4535:Question 4165:helpful. 4050:Rjwilmsi 2847:Thannks. 1835:contribs 1827:Carensdp 1823:unsigned 1706:nn123645 1666:contribs 1654:made by 1408:contribs 1250:contribs 1238:unsigned 1209:Jim no.6 1131:contribs 1119:unsigned 1043:Thanks. 984:contribs 972:unsigned 851:Rjwilmsi 836:Rjwilmsi 806:the set. 532:. Since 332:datasets 101:contribs 52:Approved 20:‎ | 6545:WT:BRFA 5805:someone 5703:Comment 5644:Request 5630:maximum 5623:Summary 5036:WP:BITE 4687:. Wow. 4485:Request 4468:Overall 4213:Glitch? 4035:Arthena 3725:Kingpin 3690:Kingpin 3646:Here's 3146:Kingpin 1971:Kingpin 1943:Kingpin 1910:Kingpin 1344:Imjayyy 1311:Imjayyy 682:MBisanz 445:ClueBot 152:Tim1357 41:WT:BRFA 6251:(talk) 5538:, and 5336:always 5320:always 5110:forces 5043:Huggle 4989:wrong. 4957:0.25%. 4798:hockey 4514:Anomie 4145:Please 3942:Ixfd64 3685:assume 3236:Ixfd64 2767:JohnCD 2630:Ucucha 2543:Kime79 2518:Please 2483:Please 2449:Please 2339:Please 2304:(talk) 2185:Ixfd64 2081:Minima 2040:nobots 1890:(talk) 1494:Lemsip 1420:Ixfd64 730:le_Jrb 633:, and 486:db-g10 473:db-g12 458:solely 313:Ixfd64 6145:KNOWZ 6108:This 5791:from 5528:stats 5418:Naomi 5299:user. 5244:three 5231:this. 5192:them. 5106:lower 4848:This 4774:Rd232 4301:edit. 4148:Stand 3710:KNOWZ 3522:) · 3185:This 3070:Quiff 2865:from 2676:these 2521:Stand 2486:Stand 2452:Stand 2342:Stand 2076:added 2003:KNOWZ 1555:again 1526:(UTC) 942:) · 897:graph 775:user? 281:slakr 236:(Y/N) 229:Yes. 224:(Y/N) 106:count 16:< 6528:talk 6418:Edit 6389:Edit 6374:Edit 6359:talk 6338:talk 6327:Edit 6291:talk 6266:talk 6240:this 6233:Edit 6217:talk 6202:talk 6187:talk 6173:talk 6149:TALK 6143:HELL 6131:talk 6117:talk 6103:Edit 6091:talk 6076:talk 6049:talk 6034:Edit 6020:talk 6012:this 6005:Edit 5991:talk 5967:talk 5945:talk 5925:talk 5917:this 5910:Edit 5878:talk 5843:Edit 5831:talk 5817:talk 5803:was 5774:talk 5759:talk 5745:Edit 5733:talk 5719:talk 5700:Diff 5669:talk 5545:The 5376:talk 5349:talk 5267:talk 5221:talk 5206:talk 5183:talk 5160:talk 5133:talk 5092:talk 5077:talk 5055:talk 5004:talk 4937:talk 4904:talk 4889:talk 4875:talk 4859:talk 4833:Cobi 4819:talk 4793:Addi 4756:talk 4716:talk 4685:this 4660:here 4641:talk 4607:talk 4591:talk 4571:talk 4550:talk 4496:talk 4372:talk 4348:talk 4330:talk 4317:Cobi 4277:talk 4237:talk 4219:this 4199:talk 4195:Gigs 4182:talk 4106:will 4085:talk 4020:talk 3993:talk 3969:talk 3946:talk 3923:N419 3882:talk 3866:talk 3842:talk 3780:talk 3740:Cobi 3729:talk 3714:TALK 3708:HELL 3694:talk 3656:talk 3614:talk 3573:talk 3555:Cobi 3544:talk 3524:@538 3519:talk 3500:talk 3484:talk 3458:talk 3443:talk 3416:talk 3400:talk 3382:talk 3362:talk 3347:talk 3318:talk 3304:talk 3289:talk 3273:talk 3255:talk 3240:talk 3216:talk 3200:talk 3165:talk 3150:talk 3136:talk 3114:Soap 3088:Soap 3060:talk 3037:talk 3022:talk 2998:Cobi 2988:talk 2972:talk 2960:here 2946:talk 2917:talk 2902:talk 2885:Cobi 2875:talk 2853:talk 2840:Cobi 2826:talk 2810:talk 2800:see 2786:talk 2771:talk 2737:talk 2733:Smjg 2714:talk 2700:Cobi 2689:talk 2666:talk 2643:talk 2620:talk 2595:talk 2580:talk 2557:talk 2501:talk 2466:talk 2432:talk 2417:talk 2397:talk 2382:talk 2374:here 2356:talk 2318:talk 2298:corn 2277:talk 2262:talk 2242:talk 2227:talk 2204:talk 2189:talk 2172:talk 2158:talk 2139:talk 2124:talk 2107:talk 2092:talk 2057:Cobi 2050:bots 2025:talk 2007:TALK 2001:HELL 1997:. — 1995:this 1988:bots 1975:talk 1961:talk 1947:talk 1941:. - 1939:here 1929:talk 1914:talk 1878:Wiki 1856:talk 1831:talk 1800:talk 1785:talk 1762:talk 1747:talk 1724:talk 1710:talk 1689:talk 1660:talk 1637:talk 1599:talk 1577:talk 1563:talk 1539:talk 1520:talk 1509:talk 1484:talk 1472:edit 1453:talk 1439:Cobi 1424:talk 1402:talk 1386:talk 1366:talk 1362:Gigs 1348:talk 1334:talk 1315:talk 1300:and 1285:talk 1270:talk 1246:talk 1213:talk 1192:talk 1178:talk 1163:talk 1145:talk 1127:talk 1091:talk 1075:talk 1052:talk 1035:talk 1020:talk 998:talk 980:talk 944:@234 939:talk 909:talk 893:here 876:talk 822:talk 748:talk 712:Cobi 698:talk 669:talk 643:talk 618:talk 603:talk 599:Gigs 589:Cobi 578:talk 563:talk 547:talk 515:talk 504:BLPs 449:and 424:talk 410:talk 406:Gigs 396:talk 376:talk 372:Gigs 362:talk 347:talk 317:talk 299:talk 241:No. 181:(s): 136:flag 116:logs 96:talk 86:BRFA 6504:Ree 6039:Or 5811:. 5795:to 5632:). 5547:FAQ 5366:, 5174:for 5125:now 4965:not 4747:and 2336:). 2294:AIR 2217:on 2148:On 1883:Dao 1674:nyb 1650:of 1617:nyb 1474:to 1435:ANN 1414:to 790:man 783:Mr. 780:-- 528:on 507:can 271:not 166:UTC 158:). 150:), 111:SUL 6559:: 6530:) 6509:dy 6361:) 6340:) 6293:) 6268:) 6219:) 6204:) 6189:) 6175:) 6133:) 6119:) 6093:) 6078:) 6051:) 6022:) 5993:) 5969:) 5947:) 5927:) 5880:) 5833:) 5819:) 5776:) 5761:) 5735:) 5721:) 5671:) 5534:, 5530:, 5498:/ 5444:2. 5415:— 5378:) 5370:. 5362:, 5351:) 5269:) 5223:) 5208:) 5185:) 5162:) 5135:) 5094:) 5079:) 5057:) 5006:) 4939:) 4906:) 4891:) 4877:) 4861:) 4821:) 4803:10 4758:) 4733:| 4718:) 4695:– 4670:– 4662:. 4643:) 4609:) 4593:) 4573:) 4552:) 4498:) 4374:) 4350:) 4332:) 4279:) 4239:) 4221:? 4201:) 4184:) 4118:. 4087:) 4022:) 4011:}} 4005:{{ 3995:) 3971:) 3948:) 3928:BH 3884:) 3868:) 3844:) 3817:ik 3808:Sh 3805:-- 3782:) 3731:) 3696:) 3658:) 3616:) 3575:) 3546:) 3526:· 3516:· 3513:X! 3502:) 3486:) 3460:) 3445:) 3437:. 3418:) 3402:) 3384:) 3364:) 3349:) 3334:, 3320:) 3306:) 3291:) 3275:) 3257:) 3242:) 3218:) 3202:) 3167:) 3152:) 3138:) 3062:) 3039:) 3024:) 3014:}} 3008:{{ 2990:) 2974:) 2962:. 2948:) 2919:) 2904:) 2877:) 2855:) 2828:) 2812:) 2788:) 2773:) 2739:) 2716:) 2691:) 2668:) 2645:) 2622:) 2597:) 2582:) 2559:) 2503:) 2468:) 2434:) 2419:) 2399:) 2384:) 2358:) 2320:) 2279:) 2264:) 2244:) 2229:) 2206:) 2191:) 2174:) 2160:) 2141:) 2126:) 2109:) 2095:) 2053:}} 2047:{{ 2043:}} 2037:{{ 2027:) 1991:}} 1985:{{ 1977:) 1963:) 1949:) 1931:) 1916:) 1887:☯ 1874:. 1858:) 1837:) 1833:• 1802:) 1787:) 1764:) 1749:) 1726:) 1712:) 1704:-- 1691:) 1676:) 1639:) 1619:) 1601:) 1579:) 1565:) 1541:) 1486:) 1455:) 1426:) 1388:) 1368:) 1350:) 1336:) 1317:) 1287:) 1272:) 1252:) 1248:• 1215:) 1194:) 1180:) 1165:) 1147:) 1133:) 1129:• 1109:}} 1103:{{ 1093:) 1077:) 1054:) 1037:) 1022:) 1000:) 986:) 982:• 946:· 936:· 933:X! 911:) 878:) 824:) 788:Z- 750:) 700:) 671:) 645:) 630:, 620:) 605:) 580:) 565:) 549:) 541:? 517:) 489:}} 483:{{ 476:}} 470:{{ 426:) 412:) 398:) 378:) 364:) 349:) 319:) 311:-- 301:) 168:) 6547:. 6526:( 6499:— 6472:. 6357:( 6336:( 6289:( 6264:( 6215:( 6200:( 6185:( 6171:( 6147:? 6129:( 6115:( 6089:( 6074:( 6047:( 6018:( 5989:( 5965:( 5943:( 5923:( 5876:( 5829:( 5815:( 5772:( 5757:( 5731:( 5717:( 5667:( 5549:. 5542:. 5502:. 5374:( 5347:( 5265:( 5219:( 5204:( 5181:( 5158:( 5131:( 5090:( 5075:( 5053:( 5002:( 4935:( 4902:( 4887:/ 4873:( 4857:/ 4853:( 4817:( 4754:( 4714:( 4639:( 4605:( 4589:( 4569:( 4548:( 4517:⚔ 4494:( 4370:( 4346:( 4328:( 4275:( 4235:( 4197:( 4180:( 4083:( 4018:( 3991:( 3975:" 3967:( 3944:( 3903: 3880:( 3864:( 3840:( 3827:) 3821:( 3814:r 3811:i 3778:( 3727:( 3712:▎ 3692:( 3654:( 3612:( 3571:( 3542:( 3510:( 3498:( 3482:( 3456:( 3441:( 3414:( 3398:( 3380:( 3360:( 3345:( 3316:( 3302:( 3287:( 3271:( 3253:( 3238:( 3214:( 3198:( 3163:( 3148:( 3134:( 3120:— 3108:— 3094:— 3082:— 3058:( 3035:( 3020:( 2986:( 2970:( 2944:( 2915:( 2900:( 2873:( 2851:( 2824:( 2808:( 2784:( 2769:( 2735:( 2712:( 2687:( 2664:( 2641:( 2618:( 2593:( 2578:( 2555:( 2499:( 2464:( 2430:( 2415:( 2395:( 2380:( 2372:( 2354:( 2316:( 2275:( 2260:( 2240:( 2225:( 2202:( 2187:( 2170:( 2156:( 2137:( 2122:( 2105:( 2089:( 2086:c 2023:( 2005:▎ 1973:( 1959:( 1945:( 1927:( 1912:( 1903:) 1899:( 1854:( 1829:( 1798:( 1783:( 1760:( 1745:( 1722:( 1708:( 1687:( 1672:( 1663:· 1658:( 1635:( 1615:( 1597:( 1575:( 1561:( 1537:( 1522:) 1518:( 1511:) 1482:( 1451:( 1422:( 1405:· 1400:( 1384:( 1364:( 1346:( 1332:( 1313:( 1283:( 1268:( 1244:( 1211:( 1190:( 1176:( 1161:( 1143:( 1125:( 1089:( 1073:( 1050:( 1033:( 1018:( 996:( 978:( 930:( 907:( 874:( 820:( 746:( 726:A 696:( 667:( 641:( 616:( 601:( 576:( 561:( 545:( 513:( 491:. 478:. 422:( 408:( 394:( 374:( 360:( 345:( 315:( 297:( 239:: 227:: 138:) 84:( 66:. 54:. 43:.

Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.

Knowledge

:Bots/Requests for approval/ClueBot NG - Knowledge

Index