Knowledge:Knowledge Signpost/2022-09-30/Recent research

791:, and while I can't recall the details of my recommendation, I am pretty sure that as an expert in that field (i.e. someone who published nearly identical research to the one described here), my conclusion would be as follows: "While it is good to understand how Knowledge Education program is done in Estonia - as vast majority of published research on this is from USA - this research is asking for 30k USD to do work that most scholars, including myself, do without any grants." Consider this: "Salary or stipend: 22.586$ ". Errr, but scholars are already employed by the university, they already get salary or stipend, and they are hired for the explicit purpose of doing such research. We are effectively giving them double salary for what they should be doing anyway, and for what most scholars I know do without receiving such a grants (You don't need any money to do "a multi-part questionnaire" (I've done many, for the cost of 0$ ), ditto for interviews (newsflash: they can be done via Zoom or such, and such tools are also available at no cost!). Then "Open access publishing costs" budgeted for 6k. Most journals I am aware of that offer OA pubishing in social sciences tend to do it for much less (just google for "cost of open access publishing", the cost is half or a third of the asked and awarded amount). Also, realistically, the cost here can be 0$ - publish without OA, just make a copy of the paper available as a pre/post print. That's 6,000 bucks we are spending for exactly no benefit to anyone (and if you disagree on official OA vs pre/post print, well, there's still the case of vastly inflated budget item here). While I think it would be good to have a fund to make reserch on Knowledge open access, I think the current edition of WRF is wasting a lot of money on dubious projects, where the costs are very, very inflated for no good reason except, well, do I need to spell it out? I am sorry, but we are paying $ 40k for a poorly laid out plan for some folks to do what they are being paid for (by their institution) anyway? This needs to be stopped ASAP. Ping 1176:(or no estimates at all) can be found in some accepted grants. We need a stricter control of this. I also wonder, will there be receipt control? At my university, I am required to prove that I actually spend the money budgeted for X on X, and RETURN the amount that wasn't used. What about our case? PS. Another point - at my university, I am required to publish the research within ~2 years or return the whole amount. What kind of accountability do we have if a grantee fails to deliver? And what are they supposed to deliver, exactly? When I apply for a grant at my institution, I am required to publish at good journals, as defined by being indexed in reliable indices. Do we have such requirements? Or will we accept an open access publication in semi-predatory journal, or a conference presentation at Wikimania, or a non-peer reviewed pre-print? What are our standards? The project I noted as controversial is promising "writing and submitting two articles for publishing them in the open access journals (e.g. Classroom Discourse)" and budgeting ~6k for that. I know that in this field (education) there are open access journals with zero fees; at the same time there are also low-impact journals in which publication is mostly inconsequential. PPS. This is related to the toothless 1010:

abused by (or, if you prefer, our budget is being wasted on) people who are effectively trying to scam WMF/us, by getting funds for stuff that is either irrelevant to Wikimedia projects or stuff that would get done for free or is alraedy paid for (people applying for grants to double/triple their salaries, which they get to do this kind of research anyway). I think Bluerasberry mentioned another dimention, that is that we have little ethical control over the studies themselves, although frankly that's the one I am least worried about. I'll finally mention something else, that another person suggested to me, which is that a possibility of corruption in the process, due to poor oversight. Given how poorly designed some of the accepted projects are, everything is possible, but bottom line is that WMF is giving out money for stuff that is of little use to the community and poorly justified. Something should be done before this gets worse and generates some serious scandal down the road. --

258:

focus on one aspect of the article’s context, the characteristics of the article writers: their motivations (to present unbiased information, fix errors, help readers understand) and their perceived domain expertise. Intriguingly, readers do not seem to consider the "wisdom of the crowd" to be a particularly salient factor when making credibility assessments about Knowledge articles: the three lowest-ranked trust components all relate, in one way or another, to the relationship between crowdsourcing and quality (search popularity, number of contributors, and number of reviewers). This finding suggests that, at least nowadays, reader trust in Knowledge is not strongly influenced by either its status as one of the dwindling-number of prominent open collaboration platforms, or its ubiquity at the top of search results.

320:"As expected, several of the existing Knowledge templates significantly influenced reader trust in the negative direction. This is unsurprising, as these templates are designed to indicate a serious issue and inspire editors to mobilize. The remaining templates, ‘Additional citations’, ‘Inline citations’, ‘Notability’, ‘Original Research’, ‘Too Reliant on Primary Sources’ and ‘Too Reliant on Single Source’ did not result in significant changes. It is possible that the specific terms used in these templates were confusing to the casual readers taking the survey. Particularly strong effects were noted in ‘Multiple Issues’ (-2.101; ‘Moderately Lowered’, p<0.001), ‘Written like Advertisement’ (-1.937, p<0.001), and ‘Conflict of Interest’ (-1.182, p<0.05)." 103: 337: 266:), a sample of survey participants were interviewed more in-depth about their previous answers, with the goal of "gain a deeper understanding into the factors that mediate a reader’s trust of Knowledge content, including but not limited to citations." Combining results from the interviews and surveys, the researchers arrive at a refined "Taxonomy of Knowledge Credibility-Assessment Strategies", comprising 24 features in three overall categories: "Reader Characteristics" (e.g. familiarity with the topic), "Knowledge Features" (e.g. its "Pagerank" or its "Open Collaboration" nature), and "Article Features" (e.g. "Neutral Tone", "Number of Sources"). 1364:

and merits that they observe with each proposal and assess whether they want to update their assessment after the discussions. After the discussion period was over, it was the job of the Research Fund chairs to make the final call when convergences was not achieved. This is in-line with the scientific review processes I have been part of or led in the past. What this means is that a reviewer may have said No to a proposal, but if they have not convinced their two other colleagues that the proposal should receive rejects from them, and if others have assessed the proposal positively, we may decide for the proposal to move to the next stage.

304:" despite the demonstrable success of Knowledge, it suffers from a lack of trust from its own readers. The Wikimedia foundation has itself prioritized the development and deployment of trust indicators to address common misperceptions of trust by institutions and the general public in Knowledge . include measuring user activity; the persistence of content; content age; the presence of conflict; characteristics of the users generating the content; content-based predictions of information quality; and many more. However, a key issue remains in translating these trust indicators from the lab into real world systems such as Knowledge." 221: 1087:

academics, I can promise you that the vast majority of that funded work—thousands of hours of effort—would never happened in the absence of external funding. Grant-funded salary has created new research resources by allowing me to hire staff or bring on graduate students. It has freed up hours that would have spent on teaching, administration, and other activities. It has allowed me and my lab to produce more Wikimedia research that I would have been able to do have otherwise. The WMF research grants program is on a smaller scale but I believe it has the potential to do something similar. —

366: 175: 110: 1356:

identified many areas for improvement (too many in fact to tackle all at once) and we have chosen a few of those to focus on and improve for the upcoming cycle. For example, Mako and I (as the Fund Committee Chairs) have prioritized to bring in dedicated Technical Review Chairs for the upcoming cycle because the load of operations on the two of us was too heavy to be repeated. This is to say that retrospectives will need to happen every year and we need to continue improving things for this and other processes.

130: 248:"Overall, respondents reported a very high level of trust in Knowledge. 88% of respondents to the first survey reported that they trusted Knowledge as a whole "a lot" or "a great deal". 73% of respondents to the second survey reported that they trusted the information in the article they were currently reading "a lot" or "a great deal" (94% in the first survey 6 ). In contrast, less than 4% of respondents in the second survey reported distrusting the information in the current article to any degree." 1360:

feedback to the applicants. On the other hand, all the research scientists in my team were already involved in providing research review to researchers in places outside of the WM Movement. At the point when my team could manage to have a dedicated person to focus on the research community I assessed that it's the right time to support the Community Resources by accepting more responsibility for research applications and also explicitly dedicating an entry point to them: Research Funds.

1119:

controls over how things are spent are fewer, and corruption, or at least the concept of, to say it bluntly, milking naive First World donors for second-third-fourth-etc. salaries by, for example, inflating costs, is not uncommon. I am sure that you yourself represent very good ethical standards, but you have to be careful when dealing with grant applications from the rest of the world - many do not share your ideals, best practices, and like, and will simply try to abuse the system.

432: 614: 90: 120: 36: 140: 100: 1213:—if I'm reading correctly, that's a million dollars donated by Knowledge readers. Can you explain what improvements this academic research has made to the Wikimedia community that is worth a million dollars to us? How would you explain the value of these projects to a small donor that read a fundraising banner and was led to believe that Knowledge is barely surviving or needs money for servers? — 212:"Through surveys and interviews, we develop and refine a Knowledge trust taxonomy that describes the mechanisms by which readers assess the credibility of Knowledge articles. Our findings suggest that readers draw on direct experience, established online content credibility indicators, and their own mental models of Knowledge’s editorial process in their credibility assessments. " 229: 1399:

leadership transitions in WMF over an extended period of time limited my capacity to engage at the extent that I would have liked to see on this topic. I communicated to the group that I needed more time due to the transitions, and my intention is to go back to this topic as I find it important to explore and develop a solution (or at least a definite answer) on this topic.

150: 1381:

amazing and I encourage those of you who are in that position to continue offering your time and expertise to the WM projects in the way that suits your particular affordances. However, it is also my responsibility to assure that we experiment with ways to support others who may not be in these positions. There is no one solution that fits everyone and that is okay.

912:

especially considering that they would interact with paid researchers at universities and companies. Research ethics takes sponsorship, and right now, there has never been either a community request for that nor a WMF offer for it. I like the idea of community ethics review, not WMF staff ethics review, but still the community needs some money.

1345:(check under Conducting Foundational Research section). I share this information with you because I find that sometimes it is helpful to step back and see what we're trying to achieve in the big picture (nurturing the WM research community) and look at the portfolio of investments that we do on top of narrowing down on specific initiatives. 788: 446: 425: 332:"The percentage of readers who had not seen the intervention completely was 48.5%. We found this surprising, as our notices (including existing Knowledge templates) were placed in a high visibility location where current Knowledge templates reside and multiple task design elements were put in place to help participants focus on them." 313:

and enables the reader to drill down to see component metrics which were contextualized to make them more understandable to an unfamiliar audience." Participants were then asked various questions, some designed to explore whether they had noticed the intervention at all, others about how they rated the trustworthiness of the content.

1077:

process. And in fact, it was discussed for the WMF research grants by a group that included you. Not everybody agrees with your opinion about whether or not funded work will/would happen in the absence of funding, either in general or in these specific cases. And nobody ever gets to know the answer unless we never fund anything.

349:"reliable increases in trust at top indicator levels This suggests that a trust indicator can provide system designers with the tools to dial trust in both positive and negative directions, under the assumption that designers choose accurate and representative mappings between indicator levels and article characteristics." 345:

They were told that it "shows the trustworthiness score of the article, calculated from publicly available information regarding the content of the article, edit activity, and editor discussions on the page", and then asked to rate the article's trustworthiness again (among other question). This resulted in

1359:

The suggestion for having a dedicated Research Fund came to me from the Community Resources team. The team used to receive funding requests for proposals that could be considered research heavy proposals, however, the team did not have access to technical reviewers who could give appropriate research

1332:

First and foremost: I kindly ask that we try hard to refrain from comments or phrases that can put a shadow of doubt on others' intentions unless absolutely justified or necessary in a particular situation. I find a word such as "scam" accusatory towards others. I feel hurt reading it as I put myself

1296:

originally academic research? So I'm just confused as to what impact we would expect these projects to have. Is it going to improve NPP? Lua? Automate some repetitive AWB activity? Suggest action points on how the community could be more welcoming to newcomers?As for the fundraising messages, the WMF

1175:

Sanity checks for individual budget items would be good, for example, one approved entry asks for 6k for open access publishing costs, a simple google query tells us the average cost of open access publishing in related fields is 1.5k-2k$ . What will happen to the other ~4k$ ? Other dubious estimates

1086:

Perhaps it is because I am from a more grant-funded part of academia than you are but I can say that I strongly disagree with your general skepticism about the using of grants and external funding for academic salaries. As someone who has overseen more than a million dollars of grant-funded salary to

252:

Survey participants were also asked about their reasons for trusting or distrusting Knowledge in general and the specific article they had been reading when seeing the survey invitation. The researchers distilled these free-form answers into 18 "trust components", and present the following takeaways.

1297:

fundraising team and foundation leadership are well aware of community objections, but generally choose not to respond to them. I raise it because if you choose to work with the research fund, you should consider whether you are happy with the money came from and under what pretenses it was given. —

1180:

which, a, allows green OA/preprints (so it is ok to publish at no cost), and b, it doesn't seem to require publication in "good" or even "mediocre" journals. I am sorry, but this ripe for abuse, people can budget thousands of dollar for OA, and then publish at no fee, and keep all the funds. This is

1143:

Fair enough. One of the stated goals of these grants are to fund research being done in places that are not (already highly-resourced) institutions in the wealthiest countries. We are already asking members of the regional committees to review and give feedback on any proposal in "their" region with

911:

If social and ethical review were a goal, then I think asking WMF to sponsor a research review committee is the most likely way to begin. Such a committee will not spontaneously appear from crowdsourcing, and even if we had a heroic team of volunteers, I think it is just too much work for volunteers

344:

In the third experiment, readers were shown articles first without and then with the newly designed trust indicator, which displayed various quantitative ratings (e.g. "Quality rating: official evaluation given by reputable editors", "Settledness: length of time since significant edits or debates").

312:

participants were shown versions of Knowledge articles modified by artificially adding warning templates (both existing ones and a new set designed by the authors, in several difference placements near the top of the page), and lastly by "a new trust indicator that surfaces an aggregate trust metric

216:

They also note that their study appears to be "the first to gather data related to trust in Knowledge, motivations for reading, and topic familiarity from large and geographically diverse set of Knowledge readers in context–while they were actually visiting Knowledge to address their own information

1376:

program in WMF. The program is designed to bring research expertise to WMF and the WM projects without direct financial investment by WMF. The program is one of the most successful programs in the Research team and has enabled us to deliver what would have been otherwise impossible to deliver given

1056:

I have never received a grant from WMF but I've used other grant funding to "buy out" of teaching classes and certain service or administrative obligations. I have paid my own salary during the June through August when I am not given a salary at all and would otherwise engage in consulting or extra

935:

Human subject research at universities requires ethical review (typically through an institutional review board). I am not saying that Meta / Facebook did anything wrong here, however, this is the first time that a big tech company has done Knowledge research like this. There will be many more such

1398:

I want to be transparent that WMF has received at least one request from one of the groups in the WM community for exploring options for ethical oversight of research conducted by WMF. We had a few good exploratory conversations with some of the members of this community a while back. However, the

1367:

With regards to oversight, I am open to receive suggestions for improvement. fwiw: One of the first things I did was to make sure a respected member of the Wikimedia research community joins me as the Research Fund chair and we make every key decision together. I'm really thankful to Benjamin Mako

1363:

Every application that was submitted during the last cycle that was not desk-rejected in Stage I received at least 3 technical reviews. When we had all accept or all reject assessments, our job was straightforward. When we didn't, we invited the reviewers to discuss among themselves the challenges

1066:

Most commonly, I will pay a graduate student or staff member to work on a grant-funded project instead of teaching or other research work they would do to make ends meet. All of these are completely normal and research universities typically have systems for tracking individual "effort" to prevent

1042:

and am doing so again this year. For several reasons, I won't speak to any of the specific proposals that were/weren't funded. I will speak to the claim the funding for academic effort is being used to increase salaries and that the work would be done without the grant. In almost all (all?) cases,

994:

repair the bugs in MediaWiki software themselves all for free. The volunteers have no official voice, the BoT is a WMF rubber stamp, and the volunteers have no funds themselves even if they had an established 'user group' - which would need incorporation as a registered charity if it were going to

257:

The four components that respondents find most salient (highest agreement) relate to the content of the article: assessments of the clarity and professionalism of the writing, the quality of the structure, and the accuracy of the information presented. The next four highest-ranked trust components

1380:

One of the 4 groups that the Research team at WMF serves is the WM Research Community and one of the asks of some folks in this community to us has been dedicated funding. I understand that some members of the research community may not need funding to conduct research on the WM projects. That is

1371:

It is my assessment that it is important for WMF to invest in nurturing the WM research community in a variety of ways, one of which is through the Research Funds. fwiw, I am also a strong supporter of the Technology Funds (to support the developer community). I do believe that in order for us to

1287:

I understood the first sentence, but okay, your "million dollars" was based on other grant funding roles.Can you link me to a page that outlines the potential impact of funded projects to the community? Where are these projects documented? From my experience—about 9 years as an editor and several

989:

want to hear (I have proof of at least one example that from about 12 years ago). They are hardly likely to fund research that will risk going the wrong way for them. They learned their lesson on that with the ACTRIAL which they paid for and which the results, despite their remonstrations, proved

762:

and to have readers scrutinise us. It is somewhat flattering that our readers have high trust rather than low trust, but each is dangerous. The level of trust I have in Knowledge is healthy skepticism. I don't take Knowledge as gospel, and if it's important for me to be confident in a fact then I

1009:

I think your concerns, while valid, are a bit different from mine. You seem to worry that WMF is wasting money on research that is biased towards what they want to hear. It's possible, since we don't know what oversight is there and who is making decisions. What I worry for is that WMF is being

1355:

This past year was the first iteration of the Research Fund. Like any other first time projects or initiatives I expect that we need to make improvements to the project over time. In July 2022, the team involved in the operations of Research Fund met for a retrospective. As you may imagine, we

1118:

To keep it short, I am sure what you say is right, the problem is that AFAIK your experiences are from "First World", institutions, where standards, ethnics consideration, and oversight are high. You may be less familiar with the "Second World" reality (not to mention the "Third World"), where

1076:

If your concern is limited to the more subtle point that "they would have done it without the grant", the best any of us can do is speculate about this for anything that is funded. It's absolutely the case that the counterfactual you are worried about is discussed as part of the grant decision

1252:

As a donor myself, I think a lot about your questions. It's far too early to know how the first round of funded projects will turn out but potential impact to community was one one of the primary criteria for evaluation. I believe the grants are all valuable use of WMF's existing resources.

324:

Four of the 11 notices newly created by the researchers also significantly affected trust: "The strongest negative effects were found in ‘Editor Disputed References’ (-1.601 points from baseline, p<0.001), ‘General Reference Issues’ (-1.444, p=0.002), ‘Tone and Neutrality Issues’ (-1.184,

143: 70: 714: 113: 411: 1067:

the kind of fraud you suggest. If you have concrete evidence of people being paid twice for the same effort, please report that to the grants team at WMF or to the research institute involved. If you do not, accusing people of fraud in a public forum seems wildly inappropriate.

377:, which provides grants between $ 2k and $ 50k "to individuals, groups, and organizations with research interests on or about Wikimedia projects across research disciplines including but not limited to humanities, social sciences, computer science, education, and law." 243:

displayed on English Knowledge readers in early 2019, asking questions such as "How much do you trust the information in the article you are reading right now?." Among the topline results, the researchers highlight that, consistent with some earlier readers surveys

280:

quality class of that article (Spearman’s Rho 0.067, n=1312, p = 0.014). This provides additional evidence that readers are able to accurately assess the general quality of the article they are reading, and that content-related factors do inform their credibility

1368:

Hill for accepting to work with me on this front last year (and this year). I consider Mako's partnership with me as one way that I can assure the volunteer Wikimedia research community has direct power and voice at the highest level of the Research Fund process.

153: 786:

I have some serious concerns about the Wikimedia Research Fund. I have been a reviewer for some projects, and in my view a lot of them are asking us to fund what is otherwise a regular activity in academia that would be done anyway. I am pretty sure I reviewed

133: 1372:

achieve our mission effectively, we must be willing to explore new options and navigate non-trivial trade-offs. As a result, you see me many times making decisions that are not at the extremes. For example, many years ago I advocated for the creation of the

453: 360: 939:

The usual route for establishing a paid process is forming a user group, requesting funds, then having annual planning. I am not keen on doing the administration, but if a research ethics committee existed, then I would join it. Thoughts from others?

291: 188: 477: 389: 1340:

at WMF is to nurture the Wikimedia research community. To that end our team is involved or leads a variety of projects (Research Fund being one of them). You can learn about the depth and breadth of these projects by reviewing our bi-annual

817:

I can't comment on any of these, as a trustee - but I am quite confident that, even though there may be many applications of all sorts, most successful ones are legit. Of course, oversight is always useful and can improve the process.

439: 353:

Interestingly, neither of the two studies about Knowledge readers' trust reviewed above appears to have been aware of the other research project's findings, even though both were at least partly conducted at the Wikimedia Foundation.

1249:. I apologize for the confusion. For context, you should know that the Research Fund just announced its call for a second year and has not distributed anywhere near a million dollars across all funded research grants put together. 273:"Although the correlation is weak, finding could indicate that readers have a higher threshold for trust when they require an in depth understanding of an article’s topic vs. learning a quick fact contained within the article." 903:

Regardless of whether the issue that Piotrus raises is a problem, the more fundamental problem is that we have no community process which would identify and address it if it were a problem. There are dozens of projects at

284:"On average, trust was highest among respondents in India and Germany and lowest in Canada and Australia, although a large variability in sample size between countries suggests caution in over-interpreting these results." 847:. Maybe those fake demands for grants are why the WMF tells the volunteers there's no money left for upgrading essential software, forcing the volunteer editors to improve the software for free. Perhaps the Trustees 374: 1253:

Questions of whether WMF's fundraising messages are in line with organizational expenditures (implicit in your final question) seem like things you should direct to the fundraising team and foundation leadership. —

418: 397: 1395:

As WMF's Head of Research, I am accountable for the ethics of research conducted by the Research team at WMF. If you have specific concerns about research conducted by the team, please reach out to me.

1043:

research grants are to institutions not individuals. For every university and/or research institution I've ever been involved with, grants used to pay salary are used to "buy" time/effort and

404: 381: 482:

This study found that a 2011 English Knowledge policy change to remove the rights of inactive administrators did not reduce the (already low) frequency of admin accounts being compromised.

908:

and hundreds more that occur but without any particular community review option. Surely some of these have raised problems over the years, but we are not tracking or discussing them.

469:

Other recent publications that could not be covered in time for this issue include the items listed below. Contributions, whether reviewing or summarizing newly published research,

263: 1333:

in the shoes of the researchers who applied for the fund: we are a community and our choice of words is an indicative of who we are and how welcoming we are towards one another.

76: 380:

This is the second edition of the research fund, whose inaugural edition had closed for submissions in January 2022. Earlier this month, the Wikimedia Foundation also publicly

729: 385: 750: 433:

From the media to Knowledge: the relationship between Chilean media news and malicious edits made in the virtual encyclopedia during the Social Outbreak of Chile in 2019.

776: 915:

Now might be time for this. Facebook / Meta recently did a major Knowledge research project which included recruiting human subjects for usability testing. See this -

703: 658: 123: 688: 601: 592: 683: 653: 638: 240: 196:

2022, "the 18th International Symposium on Open Collaboration" took place in Madrid earlier this month. While the conference had started out back in 2005 as

1193: 1166: 1131: 1109: 1310: 1275: 1226: 698: 648: 1377:

the relatively small size of our team. However, the success of this model doesn't mean that this is the only model we should experiment with or invest in.

990:

their claims completely and utterly wrong. It is unthinkable that the community volunteers should be expected to write the articles, police the content,

673: 631: 953: 1426: 1413: 663: 269:

Lastly, the paper offers some more speculative exploratory analysis results "to spark discussion and highlight potential areas of future research":

643: 625: 297: 55: 44: 1004: 811: 1022: 693: 668: 868: 830: 1144:

this in mind. If you have other ideas for how we can better checks-and-balances in this way to prevent abuse, it would be great to hear them! —

923: 1328:, Head of Research at WMF. I'd like to share some perspectives and information on my end that may help this or other related conversations: 1478: 1373: 316:

Three of the 9 existing warning templates tested produced a significant negative effect on readers' trust (at the standard p=0.05 level):

789:

meta:Grants:Programs/Wikimedia Research Fund/Using Knowledge for educational purposes in Estonia: The students′ and teachers′ perspective

734: 197: 193: 758:

This should be a concern for us as it shows low media literacy around Knowledge. We are not meant to be trusted: we are meant to be

1241:

I'm a volunteer working with the foundation to help run the research fund and was speaking about my experience with grant funding

741: 236: 21: 93: 1454: 1047:

unless the supplement somehow increases the total effort that the person is spending on their overall work for the university.

1449: 1444: 763:

need to use the article's sources or find my own, but I believe most statements unless there are clear reasons for doubt. —

781: 336: 1439: 718: 554: 518: 463: 180:

A monthly overview of recent academic research about Knowledge and other Wikimedia projects, also published as the

470: 220: 361:

Wikimedia Research Fund invites proposals for grants up to $ 50k, announces results of previous year's round

1434: 1292:

reading—I could not confidently name an academic project that has provided any value to the community. Was

1269: 1160: 1103: 924:

https://tech.fb.com/artificial-intelligence/2022/07/how-ai-could-help-make-wikipedia-entries-more-accurate/

613: 49: 35: 17: 30:

How readers assess Knowledge's trustworthiness, and how they could in the future: And other research news.

340:

The "trust gauge" designed by the authors, including the "scoring explanations" displayed in experiment 3

534:

Kuznetsov, Andrew; Novotny, Margeigh; Klein, Jessica; Saez-Trumper, Diego; Kittur, Aniket (2022-04-27).

276:"We found a (weak) positive relationship between a respondent’s trust in an article and the predicted 542:. CHI '22: CHI Conference on Human Factors in Computing Systems. New Orleans LA USA: ACM. pp. 1–17. 426:

Adapting Wikidata to support clinical practice using Data Science, Semantic Web and Machine Learning

981:, a couple of points: It's seriously time to investigate the WMF's staff ethics. Any research that 918: 1282: 1255: 1204: 1172: 1146: 1115: 1089: 373:

Until December 16, the Wikimedia Foundation is inviting proposals for the second edition of its

181: 174: 1325: 948: 309: 228: 239:(begun while one of the authors was staff member at the Wikimedia Foundation) first conducted 224:"Prior to visiting this article today, how familiar were you with the topic of this article?" 447:

Using Knowledge for educational purposes in Estonia: The students′ and teachers′ perspective

200:, focused on research about Knowledge and other wikis, this year only a single paper in the 1460: 1409: 536:"Templates and Trust-o-meters: Towards a widely deployable indicator of trust in Knowledge" 292:"Templates and Trust-o-meters: Towards a widely deployable indicator of trust in Knowledge" 8: 905: 500:"Why People Trust Knowledge Articles: Credibility Assessment Strategies Used by Readers" 189:"Why People Trust Knowledge Articles: Credibility Assessment Strategies Used by Readers" 365: 1187: 1125: 1016: 1000: 964: 941: 864: 805: 796: 725: 551: 515: 485: 201: 1303: 1219: 825: 769: 543: 507: 572:"Accounts that never expire: an exploration into privileged accounts on Knowledge" 478:"Accounts that never expire: an exploration into privileged accounts on Knowledge" 1405: 1039: 535: 506:. OpenSym '22. New York, NY, USA: Association for Computing Machinery. pp. 1–10. 205: 71:

How readers assess Knowledge's trustworthiness, and how they could in the future

919:

https://meta.wikimedia.org/Research:Machine_Learning_Assisted_Wikipedia_Editing

388:

of finalists (while inviting the community to "review the full proposal"). The

328:

There was also strong evidence for "banner blindness", e.g. in one experiment

1472: 1348:

With regards to the Research Fund specifically, I have a few points to share:

547: 511: 1246: 1183: 1138: 1121: 1033: 1012: 996: 972: 936:

data science projects, and if any cause a problem, we will not be prepared.

928: 894: 886: 860: 840: 801: 499: 1298: 1233: 1214: 1057:

teaching (US academic appointments are typically for only 9 months/year).

976: 968: 898: 890: 844: 819: 792: 764: 759: 308:

The study explored this "'last mile' problem" in three experiments where

1211:

overseen more than a million dollars of grant-funded salary to academics

571: 412:

Social and Language Influence in Knowledge Articles for Deletion Debates

232:"How much do you trust the information you are reading in this article?" 1177: 995:

manage money of its own , and who would take care of the bureaucracy?

756:

Overall, respondents reported a very high level of trust in Knowledge.

504:

Proceedings of the 18th International Symposium on Open Collaboration

163: 1388:

With regards to the ethical control I'd like to share the following:

208:". In the abstract the authors summarize their findings as follows: 1239:

I have never received (nor applied for) any grant funding from WMF.

985:

carry out appears to be done in a way that it produces the results

384:

funding decisions about proposals from this 2021/2022 edition, and

1293: 277: 533: 498:

Elmimouni, Houda; Forte, Andrea; Morgan, Jonathan (2022-09-07).

882:

Wish for WMF sponsorship of a wiki volunteer research committee

1342: 1337: 1045:

can not be used to supplement salary in the way you suggest

456:" (September 1st, 2022 - May 1st, 2023, 10,000-19,999 USD) 454:

Grounding NPOV pillar in post-censored information ecology

325:

p=0.012), and ‘Assessed as Complete’ (-1.101, p=0.017)."

570:

Kaufman, Jonathan; Kreider, Christopher (2022-04-01).

540:

CHI Conference on Human Factors in Computing Systems

497: 400:" (January 2022 to December 2022, 40,000-50,000 USD) 1178:

https://foundation.wikimedia.org/Open_access_policy

739:If your comment has not appeared here, you can try 296:This paper, presented earlier this year at the ACM 449:" (June 1, 2022 - May 31, 2023, 30,000-39,999 USD) 1470: 440:Can Machine Translation Improve Knowledge Equity 298:Conference on Human Factors in Computing Systems 262:In a third phase (detailed results of which are 204:covered such topics - but won the conference's " 1181:just an example of how abusable the system is. 569: 421:" (01.06.2022 – 31.05.2023, 40,000-50,000 USD) 428:" (1 August 2022-31 July 2023, 49,867.21 USD) 414:" (March 2022 - August 2022, 5,000-9,999 USD) 161: 435:" (April 2022 - April 2023, 5,000-9,999 USD) 407:" (June 2022 – June 2023, 30,000-39,999 USD) 206:OSS / OpenSym 2022 Distinguished Paper Award 442:" (June 2022 ~ May 2023, 40,000-50,000 USD) 929:https://github.com/facebookresearch/side 364: 335: 227: 219: 855:the WMF to spend the money not just on 742: 14: 1471: 1184:Piotr Konieczny aka Prokonsul Piotrus 1122:Piotr Konieczny aka Prokonsul Piotrus 1013:Piotr Konieczny aka Prokonsul Piotrus 802:Piotr Konieczny aka Prokonsul Piotrus 751:"Why People Trust Knowledge Articles" 54: 29: 1479:Knowledge Signpost archives 2022-09 369:Logo of the Wikimedia Research Fund 27: 612: 419:Wikidata Gender Diversity (WiGeDi) 398:The Impact of Knowledge on Science 264:still to be published on Meta-wiki 56: 34: 28: 1490: 724:These comments are automatically 386:published the full proposal texts 859:causes, but on essential ones. 851:look into this and not ask, but 173: 148: 138: 128: 118: 108: 98: 88: 735:add the page to your watchlist 563: 527: 491: 300:(CHI) opens by observing that 13: 1: 1038:I co-chaired the grants with 182:Wikimedia Research Newsletter 1311:21:45, 16 October 2022 (UTC) 1276:20:43, 16 October 2022 (UTC) 1227:23:12, 15 October 2022 (UTC) 1194:03:59, 14 October 2022 (UTC) 1167:21:58, 13 October 2022 (UTC) 1132:04:26, 13 October 2022 (UTC) 1110:20:57, 12 October 2022 (UTC) 710: 18:Knowledge:Knowledge Signpost 7: 1414:23:42, 8 October 2022 (UTC) 1336:One of the mandates of the 1288:years of "Recent research" 1023:11:24, 7 October 2022 (UTC) 1005:02:28, 7 October 2022 (UTC) 954:18:14, 6 October 2022 (UTC) 869:10:12, 1 October 2022 (UTC) 831:07:24, 1 October 2022 (UTC) 812:07:11, 1 October 2022 (UTC) 777:18:50, 3 October 2022 (UTC) 405:Slow Editing towards Equity 10: 1495: 1245:as a way of responding to 464:Other recent publications 782:Wikimedia Research Fund 548:10.1145/3491102.3517523 512:10.1145/3555051.3555052 375:Wikimedia Research Fund 1209:you say that you have 732:. To follow comments, 617: 370: 351: 341: 334: 322: 310:Amazon Mechanical Turk 306: 260: 250: 233: 225: 214: 39: 1374:Formal Collaborations 616: 576:SAIS 2022 Proceedings 368: 347: 339: 330: 318: 302: 255: 246: 231: 223: 210: 38: 728:from this article's 237:The research project 906:meta:Research:Index 605:"Recent research" → 1283:Benjamin Mako Hill 1205:Benjamin Mako Hill 1173:Benjamin Mako Hill 1116:Benjamin Mako Hill 719:Discuss this story 618: 471:are always welcome 371: 342: 241:two online surveys 234: 226: 45:← Back to Contents 40: 797:User:Bluerasberry 743:purging the cache 704:From the archives 659:Discussion report 626:30 September 2022 597:"Recent research" 57:30 September 2022 50:View Latest Issue 1486: 1463: 1324:Hi all. This is 1306: 1286: 1272: 1267: 1264: 1261: 1258: 1237: 1222: 1208: 1190: 1163: 1158: 1155: 1152: 1149: 1142: 1128: 1106: 1101: 1098: 1095: 1092: 1037: 1019: 980: 951: 946: 902: 828: 822: 808: 772: 746: 744: 738: 717: 689:Featured content 636: 628: 621: 604: 596: 580: 579: 567: 561: 560: 531: 525: 524: 495: 390:funded proposals 177: 166: 152: 151: 142: 141: 132: 131: 122: 121: 112: 111: 102: 101: 92: 91: 62: 60: 58: 1494: 1493: 1489: 1488: 1487: 1485: 1484: 1483: 1469: 1468: 1467: 1466: 1465: 1464: 1459: 1457: 1452: 1447: 1442: 1437: 1430: 1418: 1417: 1343:Research Report 1304: 1280: 1270: 1265: 1262: 1259: 1256: 1231: 1220: 1202: 1192: 1188: 1161: 1156: 1153: 1150: 1147: 1136: 1130: 1126: 1104: 1099: 1096: 1093: 1090: 1031: 1021: 1017: 997:Kudpung กุดผึ้ง 962: 949: 942: 884: 861:Kudpung กุดผึ้ง 826: 820: 810: 806: 784: 770: 753: 748: 740: 733: 722: 721: 715:+ Add a comment 713: 709: 708: 707: 679:Recent research 629: 624: 622: 619: 608: 607: 602: 599: 594: 588: 587: 583: 568: 564: 557: 532: 528: 521: 496: 492: 488: 480: 466: 363: 294: 191: 186: 178: 168: 167: 160: 159: 158: 149: 139: 129: 119: 109: 99: 89: 83: 80: 69: 68:Recent research 65: 63: 53: 52: 47: 41: 31: 26: 25: 24: 12: 11: 5: 1492: 1482: 1481: 1458: 1453: 1448: 1443: 1438: 1433: 1432: 1431: 1420: 1419: 1416: 1403: 1402: 1401: 1400: 1396: 1390: 1389: 1385: 1384: 1383: 1382: 1378: 1369: 1365: 1361: 1357: 1350: 1349: 1346: 1334: 1322: 1321: 1320: 1319: 1318: 1317: 1316: 1315: 1314: 1313: 1250: 1200: 1199: 1198: 1197: 1196: 1182: 1120: 1081: 1080: 1079: 1078: 1071: 1070: 1069: 1068: 1061: 1060: 1059: 1058: 1051: 1050: 1049: 1048: 1026: 1025: 1011: 961: 960: 959: 958: 957: 956: 937: 933: 932: 931: 926: 921: 913: 909: 874: 873: 872: 871: 834: 833: 800: 783: 780: 752: 749: 723: 720: 712: 711: 706: 701: 696: 691: 686: 684:Traffic report 681: 676: 671: 666: 661: 656: 654:Special report 651: 646: 641: 639:News and notes 635: 623: 611: 610: 609: 600: 591: 590: 589: 585: 582: 581: 562: 555: 526: 519: 489: 487: 484: 479: 476: 465: 462: 460: 458: 457: 450: 443: 436: 429: 422: 415: 408: 401: 362: 359: 357: 293: 290: 288: 286: 285: 282: 274: 190: 187: 172: 171: 170: 169: 157: 156: 146: 136: 126: 116: 106: 96: 85: 84: 81: 75: 74: 73: 72: 67: 66: 64: 61: 48: 43: 42: 33: 32: 15: 9: 6: 4: 3: 2: 1491: 1480: 1477: 1476: 1474: 1462: 1456: 1451: 1446: 1441: 1436: 1428: 1424: 1415: 1411: 1407: 1397: 1394: 1393: 1392: 1391: 1387: 1386: 1379: 1375: 1370: 1366: 1362: 1358: 1354: 1353: 1352: 1351: 1347: 1344: 1339: 1338:Research team 1335: 1331: 1330: 1329: 1327: 1312: 1308: 1307: 1300: 1295: 1291: 1284: 1279: 1278: 1277: 1274: 1273: 1268: 1251: 1248: 1244: 1240: 1235: 1230: 1229: 1228: 1224: 1223: 1216: 1212: 1206: 1201: 1195: 1191: 1185: 1179: 1174: 1170: 1169: 1168: 1165: 1164: 1159: 1140: 1135: 1134: 1133: 1129: 1123: 1117: 1113: 1112: 1111: 1108: 1107: 1102: 1085: 1084: 1083: 1082: 1075: 1074: 1073: 1072: 1065: 1064: 1063: 1062: 1055: 1054: 1053: 1052: 1046: 1041: 1035: 1030: 1029: 1028: 1027: 1024: 1020: 1014: 1008: 1007: 1006: 1002: 998: 993: 988: 984: 978: 974: 970: 966: 955: 952: 947: 945: 944:Bluerasberry 938: 934: 930: 927: 925: 922: 920: 917: 916: 914: 910: 907: 900: 896: 892: 888: 883: 880: 879: 878: 877: 876: 875: 870: 866: 862: 858: 854: 850: 846: 842: 838: 837: 836: 835: 832: 829: 823: 816: 815: 814: 813: 809: 803: 798: 794: 790: 779: 778: 774: 773: 766: 761: 757: 745: 736: 731: 727: 716: 705: 702: 700: 697: 695: 692: 690: 687: 685: 682: 680: 677: 675: 672: 670: 667: 665: 662: 660: 657: 655: 652: 650: 647: 645: 642: 640: 637: 633: 627: 620:In this issue 615: 606: 598: 586: 577: 573: 566: 558: 556:9781450391573 553: 549: 545: 541: 537: 530: 522: 520:9781450398459 517: 513: 509: 505: 501: 494: 490: 483: 475: 474: 472: 461: 455: 451: 448: 444: 441: 437: 434: 430: 427: 423: 420: 416: 413: 409: 406: 402: 399: 395: 394: 393: 391: 387: 383: 378: 376: 367: 358: 355: 350: 346: 338: 333: 329: 326: 321: 317: 314: 311: 305: 301: 299: 289: 283: 281:assessments." 279: 275: 272: 271: 270: 267: 265: 259: 254: 249: 245: 242: 238: 230: 222: 218: 213: 209: 207: 203: 199: 195: 185: 183: 176: 165: 155: 147: 145: 137: 135: 127: 125: 117: 115: 107: 105: 97: 95: 87: 86: 78: 59: 51: 46: 37: 23: 19: 1422: 1323: 1302: 1289: 1254: 1242: 1238: 1218: 1210: 1145: 1088: 1044: 991: 986: 982: 965:Bluerasberry 943: 881: 856: 852: 848: 785: 768: 755: 754: 699:CommonsComix 678: 649:In the media 632:all comments 584: 575: 565: 539: 529: 503: 493: 481: 468: 467: 459: 379: 372: 356: 352: 348: 343: 331: 327: 323: 319: 315: 307: 303: 295: 287: 268: 261: 256: 251: 247: 235: 215: 211: 192: 179: 164:Tilman Bayer 94:PDF download 1461:Suggestions 793:User:Pundit 726:transcluded 674:Serendipity 202:proceedings 144:X (Twitter) 1425:. You can 1421:It's your 1406:LZia (WMF) 1243:in general 1189:reply here 1127:reply here 1040:LZia (WMF) 1018:reply here 843:concerns, 807:reply here 760:verifiable 486:References 82:Share this 77:Contribute 22:2022-09-30 1455:Subscribe 730:talk page 664:Interview 382:announced 1473:Category 1450:Newsroom 1445:Archives 1423:Signpost 1290:Signpost 839:I share 644:In focus 595:Previous 217:needs." 134:Facebook 124:LinkedIn 114:Mastodon 20:‎ | 1427:help us 1247:Piotrus 1139:Piotrus 1034:Piotrus 975:, and 973:Piotrus 897:, and 895:Kudpung 887:Piotrus 841:Piotrus 694:Gallery 669:Opinion 198:WikiSym 194:OpenSym 1299:Bilorv 1234:Bilorv 1215:Bilorv 977:Pundit 969:Bilorv 950:(talk) 899:Bilorv 891:Pundit 857:worthy 845:Pundit 821:Pundit 799:... -- 765:Bilorv 154:Reddit 104:E-mail 1440:About 1326:Leila 849:could 827:utter 392:are: 16:< 1435:Home 1410:talk 1305:talk 1294:ORES 1221:talk 1001:talk 987:they 983:they 865:talk 853:tell 771:talk 603:Next 552:ISBN 516:ISBN 278:ORES 992:and 544:doi 508:doi 162:By 79:— 1475:: 1412:) 1404:-- 1309:) 1225:) 1003:) 971:, 967:, 893:, 889:, 867:) 795:, 775:) 593:← 574:. 550:. 538:. 514:. 502:. 1429:. 1408:( 1301:( 1285:: 1281:@ 1271:๛ 1266:o 1263:k 1260:a 1257:m 1236:: 1232:@ 1217:( 1207:: 1203:@ 1186:| 1171:@ 1162:๛ 1157:o 1154:k 1151:a 1148:m 1141:: 1137:@ 1124:| 1114:@ 1105:๛ 1100:o 1097:k 1094:a 1091:m 1036:: 1032:@ 1015:| 999:( 979:: 963:@ 901:: 885:@ 863:( 824:| 804:| 767:( 747:. 737:. 634:) 630:( 578:. 559:. 546:: 523:. 510:: 473:. 452:" 445:" 438:" 431:" 424:" 417:" 410:" 403:" 396:" 184:.

Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.