421:
impractical for most smaller wikis to generate. It may be possible to use the dataset generated from
Knowledge on your wiki, if the content and vandalism trends are sufficiently similar. To test this for your wiki, you will need to at least come up with a trial dataset - say, about a hundred constructive edits and fifty vandalism edits, at a minimum. More is better. With a trial dataset, you can evaluate whether or not the Knowledge training set can be used on your wiki. If your wiki is very large and has a sufficient number of edits to generate a full training set, then this is preferable, as it can then be personalized to your wiki's content and trends. However, generating the necessary datasets is up to you. It's important that they be random, and a representative sampling (i.e., unbiased). In addition to this, the trial dataset should be random, to accurately calculate a threshold and gauge effectiveness. You will also need a Linux/UNIX system to run the bot on, and sufficient knowledge to compile and install the various dependencies.
287:
is below an acceptable level, the bot can legitimately operate as an asset to
Knowledge, and improvements to the bot will result in increased vandalism catch rate. Additionally, even humans make mistakes – bots may even make proportionally fewer mistakes than humans. Bot mistakes are often noticed more because there's a greater raw volume of them (because the bot makes many more edits in total).
44:
relationships between the statistics and the output. These relationships are not always apparent to a human. A human usually looks primarily at the meaning of content, while ClueBot NG looks at a large set of numbers. It's usually very difficult or impossible for a human, just looking at that set of numbers, to figure out why ClueBot NG did what it did.
274:
review the archives to make sure your idea has not already been discussed, and also keep in mind that we do not have the time to both manually sort through chaotic freeform reporting and improve the bot. If you do post on the talk page, be sure to actually suggest an alternative method – complaints without suggestions do not help us improve anything.
420:
Before trying to set it up, you need to consider a few things. Most importantly, ClueBot NG learns what is vandalism based on a dataset, which often needs to be very large to be effective. To be effective on
Knowledge, it needs at least 10,000 constructive edits and 10,000 vandalism edits. This is
391:
wrote the core (vandalism detection algorithm) of ClueBot NG. Crispy and NaomiAmethyst were part of the same organization called ClueNet, which is the origin of both bot names. In light of the new core, and greatly increased edit rate, NaomiAmethyst largely refactored the ClueBot interface code and
269:
We used to do this, and the result was chaotic. Without explicit directions, users would leave malformed responses, responses without the required data (such as the link to the edit in question), and unrelated comments. Even with templates and comments to guide users, the result was a mess that was
236:
The false positive (FP) rate calculations do not involve the number of reported false positives. A series of edits, known as the trial dataset, are used. These edits are known to be correct (verified by humans) and random (this is important). They are run through the bot offline, and these results
273:
If you believe the instructions for using the false positive reporting interface are unclear, feel free to modify and improve them. If you dislike the entire concept of the false positive reporting interface in general, feel free to suggest an alternative method on the talk page. But first, please
260:
In our spare time, we manually review false positives. If we come across a false positive that may be able to be prevented by some method other than dataset expansion, we look into changing the code. If we see a false positive that occurred due to some reason that has not already been addressed in
286:
be room for improvement. As the bot runs, people contribute to the dataset, and it learns more and more. But, even as it learns and improves, the number of false positives won't necessarily decrease – see the above section on why ClueBot NG has false positives. As long as the false positive rate
107:
We use some edits that are automatically generated, but this is far from ideal, for two reasons. First, generated edits are always biased to some degree, depending on whatever statistics are used to generate them. Second, generated edits are often inaccurate, and small inaccuracies in the dataset
159:
ClueBot NG works by generating a probability that a given edit is vandalism. Edits with a probability above a certain (threshold) score are considered vandalism. The higher the threshold is set, the fewer false positives there are, but also the fewer vandalism edits are caught. To catch a large
144:
Anti-vandal bots do not perform precise, exact work like most other bots do. They act more like humans, with most edits correct and good, but a small percentage of mistakes. Bot edits show up as (unflagged) human edits so they can be reviewed for possible mistakes if necessary, like other human
43:
ClueBot NG works by generating a probability that a given edit is vandalism. Edits with a probability above a certain (threshold) score are considered vandalism. It's often difficult to tell exactly what the origins of a score are. ClueBot NG examines statistics from the edit, and calculates
47:
In many cases, this leads to correct classifications that could easily be missed by a human, or extremely rapid reverts where it could take a human some time to do research. On the flip side, it also means that it can be very difficult to determine why a false positive occurred. For more
98:
ClueBot NG is different from other anti-vandal bots in that it is not preprogrammed to recognize vandalism. Instead, it learns what is considered vandalism by reviewing a very large number of both vandalism and constructive edits. This very large set of edits is called the dataset.
380:
There is very little relation, besides the name. The core and algorithms used are written by different people, and use entirely different concepts. The only code shared between ClueBot and ClueBot NG is the interface to
Knowledge, and even that was refactored for ClueBot NG.
210:
If you feel you have a compelling reason that the false positive rate is too high, and your reason has not already been addressed on the talk page (please look through the archives – this topic has already been discussed at length several times), post your concerns
84:
The bot handles quoted text differently from regular text. Simply quoting a range of text is not enough to "fool" the bot if the edit is vandalism, but for constructive edits, some sometimes-acceptable bad words inside of quotes may be ignored.
215:
on the talk page. If your concerns have already been addressed by past discussions, you suggest an unreasonable false positive rate, or you do not suggest a false positive rate at all, your post will likely not receive a thorough response.
202:
to make sure you fully understand their role in the bot's operation, and the implications of adjusting the rate. Also keep in mind that the set false positive percentage is a maximum, and actual rate will probably be less due to
140:
Since anti-vandal bots are doing a steady stream of edits that would otherwise (usually and eventually) be done by a human, unflagged edits do not increase the volume of edits that show up in a feed, nor do they increase
364:. Before posting, please make sure your suggestion is realistic, practical, and makes sense. Also, please glance through the archives to make sure the suggestion has not already been submitted and/or implemented.
351:
If you have a complaint about the bot's operation, and you do not have a helpful, useful or practical suggestion about how to solve the problem, we would prefer not to hear your complaint.
257:
We submit them to the review interface for verification, then add them to the dataset. This should improve bot operation as long as it does not introduce significant dataset bias.
413:
Setting up ClueBot NG can be a complex process. You'll need to talk to the development team for instructions and help with setting it up. To talk with the dev team, join the
367:
We try to respond to and potentially implement helpful suggestions as soon as possible, particularly if they have a real chance of improving the bot's operation.
133:
Anti-vandal bot edits usually aren't tagged with the bot flag. This is intentional, and is not specific to ClueBot NG – it applies to all anti-vandal bots.
64:
that not only determines optimal weights, but can also discover more complex patterns and relationships. There are no static weights that can be modified.
345:(this is very unlikely, as machines do not reprogram themselves), and the bot is causing severe problems, you may use the emergency shut-off.
231:
277:
195:
section. It has jumped around since the start, at times being 0.5% (1 in 200), 0.25% (1 in 400), and 0.1% (1 in 1000).
55:
270:
very difficult and time-consuming to review by hand, and impossible to automatically scan to import into the dataset.
264:
348:
If you feel threatened by a bot doing a human's job, you can build a time machine and go back in time a few decades.
224:
The false positive rate is the percentage of not-vandalism edits that are incorrectly reverted as vandalism. It is
232:
Many people probably don't even report false positives, so how can you be sure the false positive rate is accurate?
199:
169:
278:
If there's room for improvement, why don't you shut down the bot until false positives are reduced or eliminated?
246:
165:
191:
ClueBot NG's false positive rate is set by the operators at runtime. The current setting can be found in the
117:
305:
301:
204:
392:
wrote the currently-used dataset collection scripts, as well as the review interface, for ClueBot NG.
361:
335:
61:
56:
Why don't you decrease/increase the weight of exclamation points/shouting/<Insert Metric Here>?
265:
Why don't you just use a page where users can post freeform responses for false positive reporting?
237:
form the basis of the FP rate calculation. This ensures that the FP rate is an accurate maximum.
73:
116:
We need people to manually review edits for the dataset. If you'd like to help out, please see
414:
334:
If you have found a bug in the bot (other than false positives), please leave a note on the
192:
161:
186:
181:
8:
79:
384:
38:
327:
If you have a problem with the bot's false positive rate or reporting interface, see
76:
to automatically generate a list of words with precise statistical probabilities.
388:
342:
27:
17:
387:
wrote the original ClueBot in its entirety, and maintained it for three years.
282:
Due to ClueBot NG's algorithm using a dataset instead of set rules, there will
300:
We always appreciate hearing that we do a good job. Please add praise to the
198:
Before complaining about the false positive rate, please read the section on
408:
341:
If the bot is not operating within the expected parameters as stated on the
219:
252:
160:
amount of vandalism, the threshold must be set at a level where there are
187:
I think ClueBot NG has too many false positives. What do I do about it?
375:
295:
128:
80:
What happens if someone legitimately uses bad words inside of quotes?
102:
30:. Please make sure your question is not listed here before asking.
182:
Why did ClueBot NG classify this edit as vandalism or constructive?
164:, but some, false positives. More information is available in the
39:
Why did ClueBot NG classify this edit as vandalism or constructive?
321:
261:
previous discussions, we usually leave a comment to that effect.
60:
ClueBot NG does not use weights. It uses an algorithm called an
311:
355:
154:
395:
316:
Complaints are filed according to the following criteria:
93:
409:
How do I run ClueBot NG on my own MediaWiki installation?
360:
We love helpful suggestions, and they can be left on the
320:
If you have a suggestion how to make the bot better, see
220:
What is the false positive rate and how is it calculated?
253:
What do the developers do with reported false positives?
175:
240:
26:
This page contains
Frequently Asked Questions about
376:
How does ClueBot NG relate to the original ClueBot?
296:
I love ClueBot NG! How can I show my appreciation?
228:the percentage of bot reverts that are incorrect.
129:Why don't ClueBot NG's edits show up as bot edits?
103:Why don't you automatically generate the dataset?
67:
312:I have a complaint. Where can I register it?
356:I have a suggestion. How do I let you know?
72:There is no bad words list. The bot uses
155:Why does ClueBot NG have false positives?
304:. Barnstars and other awards go on the
108:can cause large decreases in accuracy.
94:What is the dataset and how is it used?
14:
328:
49:
176:Why did this false positive happen?
23:
149:
111:
24:
430:
400:It stands for "Next Generation".
241:How do I report a false positive?
213:and suggested false positive rate
33:
136:There are two reasons for this:
403:
338:and we'll look into fixing it.
13:
1:
396:What does the "NG" stand for?
68:Where is the bad words list?
7:
290:
10:
435:
370:
88:
62:artificial neural network
123:
205:post-processing filters
74:Bayesian classification
48:information, see
426:
434:
433:
429:
428:
427:
425:
424:
423:
411:
406:
398:
378:
373:
358:
329:False Positives
314:
298:
293:
280:
267:
255:
243:
234:
222:
200:false positives
189:
178:
170:false positives
157:
152:
150:False Positives
131:
126:
114:
112:How can I help?
105:
96:
91:
82:
70:
58:
50:False Positives
41:
36:
22:
21:
20:
18:User:ClueBot NG
12:
11:
5:
432:
410:
407:
405:
402:
397:
394:
377:
374:
372:
369:
357:
354:
353:
352:
349:
346:
339:
332:
325:
313:
310:
297:
294:
292:
289:
279:
276:
266:
263:
254:
251:
242:
239:
233:
230:
221:
218:
188:
185:
177:
174:
156:
153:
151:
148:
147:
146:
142:
130:
127:
125:
122:
113:
110:
104:
101:
95:
92:
90:
87:
81:
78:
69:
66:
57:
54:
40:
37:
35:
34:Core Algorithm
32:
15:
9:
6:
4:
3:
2:
431:
422:
418:
416:
401:
393:
390:
386:
385:NaomiAmethyst
382:
368:
365:
363:
350:
347:
344:
340:
337:
333:
330:
326:
323:
319:
318:
317:
309:
307:
303:
288:
285:
275:
271:
262:
258:
250:
248:
238:
229:
227:
217:
214:
208:
206:
201:
196:
194:
184:
183:
173:
171:
167:
163:
143:
139:
138:
137:
134:
121:
119:
109:
100:
86:
77:
75:
65:
63:
53:
51:
45:
31:
29:
19:
419:
412:
399:
383:
379:
366:
359:
315:
299:
283:
281:
272:
268:
259:
256:
244:
235:
225:
223:
212:
209:
197:
190:
179:
158:
135:
132:
115:
106:
97:
83:
71:
59:
46:
42:
25:
415:IRC Channel
404:Other wikis
306:awards page
302:praise page
141:clustering.
193:statistics
172:sections.
166:algorithms
28:ClueBot NG
362:talk page
343:user page
336:talk page
291:Feedback
162:very few
371:Origins
89:Dataset
389:Crispy
284:always
145:edits.
322:below
124:Edits
16:<
247:here
180:See
168:and
118:here
245:Go
226:not
417:.
308:.
249:.
207:.
120:.
52:.
331:.
324:.
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.