Knowledge talk:Identifying reliable sources (medicine)/Archive 14

580:, i do appreciate you revisiting my question. I see the points you're making about the specific language i wrote. To clarify, "in many cases" meant "in the case of many congeners of PCBs" and the levels cited were averages of those whole categories, like former factory workers. So that could be made more clear with the language. In this case, i was using it solely for measurements on levels, not interpreting it as body burden or effects resulting, or anything like that. Simply to show exposure levels as reflected by amounts in serum. The source is available for others to check, of course. I'm not trying to right great wrongs, but i am hoping to get some more details about specific incidents included in this article, so it better reflects the realities of the contamination events. My main question was, what is the ultimate "spirit of the law" about the MEDRS guidelines? Is it to protect against quack claims about medical etiologies, or is it to prevent data like this which relates to human health but is not interpretive, and which relates to a historical event, from being included in Knowledge? 772:, what i see in the first paragraph of the guideline is a reference to medical advice, and then "health information", "biomedical information", and "current medical knowledge". That, to me, implies that the purpose of MEDRS is to be very careful about any information that anyone could construe as medical advice. However, on the current question, i was asking if it's ok to use toxin levels from a PCB contamination incident in Italy, using only a primary source to support the mention of the toxin level readings. That was the text of the article which i intended to support with a primary research article in Chemosphere. I take the guideline as stating that ideally, everything related to medical interpretation that may be construed as advice to be sourced to secondary sources. But nobody ever doubts that avoiding exposure to PCBs is a good idea, so this could not be construed as providing advice. It's to tell a history. It's not science, but rather social history. 1112:

articles from time to time. A paper on a potential causal pathway for a disease, for example, would be included in a review article fairly soon, if it were valid and worthy. But, a paper on serum levels of people exposed to PCBs in a historic incident may not be. There may be no forthcoming review article about PCB contamination events because it's not the sort of topic that gets this kind of periodic review. It's more historical. I would love to see some acknowledgement of this distinction, or even some disagreement to this point. I'd like this point addressed, because it's the crux of this discussion, in my mind. I would like to be able to include data about historical events regarding human health even if they're not included in a review article because this kind of study does not get reviewed very often.

3393:

more of a placebo benefit than a U.S. citizen who drinks the same formula and doesn't have the same cultural understanding of herbs and might not believe in it? If you receive more of a placebo response you are also receiving more of a total therapeutic response, and if studies fail to address issues such as these, how can we possibly red-flag a specific nation's research at the guideline level? There may be other, legitimate reasons for discrepancies in research findings other than simply fraud or bad science which we imply is the case with guidelines like this. And why are we discussing this as systemic bias when it is a publication bias issue? To me, systemic bias is exactly what we're doing when we undervalue or overvalue research or ideas from one area of the world, like we're doing here.

995:

Brescia, Italy while the factory was there making PCBs and disposing of the waste badly in the dirt", and then repeatedly demanded a response. You haven't gotten a reply on etiology because MEDRS says nothing about etiology and little about disease. Its scope is much broader. When you characterize a "simple measurement of PCB levels in certain populations" you betray a great deal of faith in the capabilities of the researchers involved. Populations are rarely "certain", and measurements on blood chemistry in those populations are rarely "simple". How much of the veal did these farmers consume? Over what time period? Over what geographic area did the animals graze? Did the animals consume surface or well water? The humans? What time elapsed between last exposure and testing?

561:. These reviews can give us a sense of which primary source findings matter, how sound they are, how to interpret them, and whether they are contradicted by other primary findings. As to your direct example, you say "Research on the adult population of Brescia" which might imply to our readers that you were representing the sum of all knowledge about that population rather than one paper. Does "in many cases" imply 40% or 4% or 4 people? How many had similarly high levels of dioxins? Did the body burden correlate to duration of exposure? Did it correlate to the proximity to the food contamination? What other anomalous exposures did that population receive? We have to rely on published experts to select the significant things to say. We cannot credibly do our own review. 933:

preferable when available. I also know the guideline about MEDRS wanting a review article from the last 5 years, preferably. But i also know that this is largely for etiology of disease and health because of the extreme need for accuracy on any attribution of causal connections relating to human health. The historical data on a contamination event is often studied within a few year of the event, or at a certain point in time and then not again, because it's already been done. And it's also not an etiology thing. It's a measurement data thing. It's a simple measurement of PCB levels in certain populations exposed to contamination. I would like your admission on these points, one by one, or denial and reason behind it.

2062:

people are X, it might be okay if we say "this individual black person is X based on these criteria", but even then we had better be cautious because we might rightfully be accused of being not very PC. To not take the same cautions regarding research is wrong and the Knowledge community at large will really be upset if they find out these are the proposed new guidelines. And don't forget some research that comes from China has positive results simply because they have different testing methodologies resulting from their unique ethics. It is systemic bias if we ever refuse to include research from other cultures and we should tag articles where this is taking place.

397:

specific human health etiologies are carefully vetted, but i don't think it should apply to things like this paper that presents historical measurements of PCB contamination levels in people who lived in Brescia, Italy while the factory was there making PCBs and disposing of the waste badly in the dirt. I don't know if i'd find another source that would give these measurements that would qualify, if this one doesn't, and therefore this sentence that i think is important and surely verifiable, would be struck.

140:

impact factor alone. At that point, you need to consider whether the idea being proposed is fringe and reject it from a weight perspective. Another way you could tackle things (though I rarely see this used here) is to look at the impact factors of all the journals being cited for reviews. If you're dealing with 10s for most journals, but a review with a unique or potentially fringe idea is at a 1, there's a good chance it's an extremely minority viewpoint there if there's even one worth mentioning at all.

414:

happens to be an editor with a chip on their shoulder who doesn't want any bad information about chemical companies to be seen, and so uses MEDRS to revert edits and disappear texts into a memory hole? I would like to know people's interpretations about the scope of MEDRS -- to what it applies and to what it does not. Like anything having to do with levels of a toxin in human beings, does that absolutely have to be sources as per MEDRS or could it be a basic, solid primary research article like

31: 3044:. Publication bias should be de-emphasized since the article doesn't even study it, and we should find a better way to address the problem as a whole. Is it best done by adding the six factors as mentioned in the article? I don't know, but perhaps additional sources could serve as enforcement. The Vickers article is discussing these factors as possibilities, but I think it's rather good discussion they're having. For example, when it comes to explanation No. 1 (the " 1305:

worked pretty much like a first-year chemistry lab: find out what the acceptable answer is, draw the curve, plot the points, and, last of all, take your measurements and make sure they line up with what the teacher wants. That doesn't mean that all Russian and Chinese research publications are as bad now, and we shouldn't be presenting this 17-year-old primary study as if it said anything useful to tell us about recent research work.

1832:

having trouble finding a direct link to it. In any case, this phenomenon is a real source of concern in the scientific world, as the above sources demonstrate. Now, I don't think we need to write anything into our guidelines and policies to address it—and I don't really want this to become another front in the acupuncture/TCM wars—but it is important to be aware that this is a real concern and not simply editorial speculation.

938:

address each point until completion, but these dialogues do not go that way and tend to be one after another topic and never completion on one claim, and then to build up not to the reality but to the appearance of a preponderance, and that's not cool. That's really a sort of bulldozer approach to dispute resolution and it's not inherently just or correct or leading to good results. It's more like a herd attack.

2181: 478:? I don't get that sense. What i am asking is, what is included in the scope of "biomedical" and isn't there a danger that we will lose ability to provide very useful information that is found nowhere else, but is *not* making a synthesis, but only providing data that will probably not be published in a "secondary source". So, if you can please engage my question and give your thoughts, i would appreciate it. 2552: 3431:, although I'm unaware of sources substantiating an effect with equivalent methodological rigor as we find in studies of Chinese sources. The purpose of MEDRS is to provide guidance as to when a source is reliable, and studies of mythology-based medical systems from countries where that particular mythology is widespread don't have the same weight as when performed by outsiders.— 228:

have done it here sometimes in afds on the notability of particular journals. It is not a mechanical operation, but requires judgment, and a knowledge of the pattern of the literature in the subject, and the other journals. There is no mechanical way of doing this; there is no shortcut. Those shortcuts sometimes tried by academic administrators have no scientific validity.

3054:

at acupuncture conferences, does not suggest any considerable number of negative results published in non-Medline journals. For example, of the many hundreds of trials reported at the third World Conference on Acupuncture , we were unable to locate any studies originating in East Asia that showed acupuncture to be equal or inferior to a control procedure.

814:

balance and weight. I have pruned some of the longer descriptions, and expanded some of the shorter, or absent ones. I am paying attention to balance and weight. It's not synthesis, because it's description of PCB exposure, and we already accept that exposure or body burden of PCBs are not a good thing. I just wanted to gain an interpretation of

3490:

light specific cases/questions for easier reference? We don't absolutely need something in MEDRS itself, but just showing that there's consensus for a general course of action (be cautious about Chinese sources in this topic) seems good for a small bulleted list where we could may include other commonly discussed specific topics. Thoughts?

402:

microbiome. Things like this have nothing to do with suggesting treatments or medicines or practices that would in any way endanger humans if the information is wrong. These are about documenting chemical spills and dynamics about pesticides that may help our general health in the future by explaining what has happened in the past.

3362:'m not 100% sure something so topic-specific should be in MEDRS the guideline but it needs to be documented somewhere, maybe in a FAQ at the affected articles. Or possibly an essay that could be linked to when necessary when a relevant discussion arises at an article Talk page. Zad68 14:28, 12 May 2015 (UTC) 2124:, I don't doubt at all that there might be serious problems with the Chinese studies. And that's the very reason we are implementing statistical methods instead of mere speculation. Statistics, BullRangifer, statistics. Unfortunately I don't have access to the full text, but so far the conclusions state: " 3489:

I generally like the most recent (#4) proposal in concept, except I agree with Sandy in that I'm iffy about putting specific cases like this into MEDRS. A possible thought I had is that since specific cases may not fit on the main guideline, page, how about making a FAQ for the talk page here to high

3449:

I don't see anyone suggesting politeness is an issue. Perhaps I've missed that. There is a fundamental point that concerns all RS, and that is that sources are viewed per the content they are designed to support or content is reliable per the sources chosen to support them. Generalizations on sources

3059:

I guess there is some thinking to be done in order to address these in an appropriate manner and to avoid representing them as too much of "carved in stone" kind of truths. I believe, however, that something can be worked out. As I originally pointed my remarks to the wording number one (the only one

2533:

Does the source urge editors to caution with respect to Chinese sources? Yes, it definitely does. Is the source a study on publication bias? No, it's not. As I have said, I don't doubt the questionable nature of Chinese sources, but I am saying that the article doesn't study publication bias. There's

2509:

and proper use of sources. As I have already said earlier, I don't doubt the questionable reliability of Chinese sources. I just got my hands on the source today and indeed, the Vickers article does not study publication bias, but deals with the reliability of those sources (China, Japan, Russia/USSR

1336:

The systemic bias WAID complains about is our systemic bias for quality research. This content reinforces that legitimate bias. By not warning against the inclusion of shoddy research, the balance tips away from quality research toward shoddy research in favor of TCM and acupuncture, which would suit

1332:

There is no such assertion. We are just doing what the research recommended. Note that the wording is lifted, with some alteration, directly from the review: "Researchers undertaking systematic reviews should consider carefully how to manage data from these countries." That's good advice for us. This

994:

You started off with "I think MEDRS exists to make sure claims about medicines and treatments and specific human health etiologies are carefully vetted, but i don't think it should apply to things like this paper that presents historical measurements of PCB contamination levels in people who lived in

896:. Your alternative of "...were found to be..." (as distinct from "...were reported as...") implies that the primary reports were accurate, significant, correctly reported, and never retracted. We do not have evidence that this is the case, which is exactly why we require current secondary sources, to 813:

Good point. In this case, the primary source provided the information about levels of PCBs in the bodies of people who were exposed in Brescia, Italy by working or living near the factory. In that article, there is a section about PCB contamination sites around the world, and i am paying attention to

594:

The friction usually comes when there is only one study or a handful of studies making a claim, that is not accepted as legitimate by the majority of professionals in the field. Consider the vaccine-autism lie: it was ignored as stupid until it became impossible to ignore its effect on public health,

528:

on this, claiming that it's a biomedical claim that falls under the scope of MEDRS and therefore i need to find a MEDRS-qualified source or else the text that i included will be removed. Is that how we want Knowledge to work? Please consider carefully, is this a MEDRS claim that would affect people's

396:

you see a user saying that my source on PCB levels in people who lived with the PCB factory in Brescia, Italy was cited as insufficient using MEDRS. Is this good practice, or is it going overboard? Does MEDRS apply to this? I think MEDRS exists to make sure claims about medicines and treatments and

139:

Essentially, an impact factor of zero and very near it is our main red flag of a shoddy journal where we wouldn't use it. Once we start getting to the 0.5 region and above, it's highly field dependent what's considered reputable, and we aren't really in a position as editors to reject a source due to

3392:

has argued, the standard here must be supported by iron clad statistics that have exhausted all other possibilities why there is a discrepancy in research findings. For instance, has any of this research tackled the idea that if a Chinese person drinks "TCM herbal formula X" that person will receive

3053:

Possibly, negative trials originating from eastern Europe and Asia are found solely in non-Medline journals. We believe that this is unlikely; Medline might be expected to be a conservative source of information on unconventional therapies. Moreover, anecdotal evidence, such as the results presented

2383:

do not hat my comment, thanks. JS check your email, as i mentioned on my talk page. it is your responsibility to get your hands on sources; continually asking if there is evidence about X when you have been provided sources goes beyond bad scholarship to bad faith discussion. i won't be responding

1690:

This is indeed a concerning state of affairs but it does appear to be well-supported that it's a problem. I'm not 100% sure something so topic-specific should be in MEDRS the guideline but it needs to be documented somewhere, maybe in a FAQ at the affected articles. Or possibly an essay that could

823:

is the section of the article about Brescia. I wanted to remove the sentence "The values reported..." and "As a result..." and re-work the section a bit, but when i did begin, i added the sentence "Research on the adult..." and then got this "medical citation needed" tag for that. I thought that was

818:

to know whether a primary source is decent for including a fact like this. Often, a review article will mention a study, with some kind of interpretation or synthesis, but it won't cite exact data like this, and in the case of Brescia contamination, i thought this was good info and reads well in the

702:

Ok, according to you, i will have to find a review article for every claim relating to biochemistry and humans. I probably can live with that. But if i find a relevant review that confirms the merits of a primary source, can i use specific details from the primary source that may not be mentioned in

227:

For a particular instance of a challenged references, certainly there is an easy way to do that--for the paper whose degree of reliability in is in question, one looks at the citations in google scholar (or ISI or Scopus). Fo rthe general evaluation of a journal, it's more complicated; I and others

143:

Is there may a single source being discussed that could be used as an example in this discussion? Once you get over zero, we really can't be going by impact factor alone, but it could very well play a role in determining weight. In medical fields though, my gist of the topic seems to be that 1's and

2102:

at the research and sources we have presented to you above. It is not editorial speculation which you are rejecting, but reliable sources and a review of the literature showing the current miserable state of affairs. Much of the literature is outright fraudulent. You even refuse to accept the exact

2061:

The other day DrChrissy questioned whether or not this line of thought was racist and I can definitely see his reason for asking the question. It does seem that we could be stereotyping if we ever ignore research simply because of its author's country of origin. Just as we would never say all black

1355:

one of the two sources about chinese publishing is dated to 2014, and there is an update to the Vickers review dated 2014 written by (ahem) Chinese people. Reporting on systemic problems is not a reflection of systemic bias; that label is inaptly applied, i think. I would support a revision to make

1304:

While the 1998 study was limited to acupuncture, it's conclusions are outdated (it's based upon the three decades of papers, ending two whole decades ago) and it's not at all clear that these societies were treating TCM any differently from any other subject. Communist research frankly sucked. It

1111:

This is all very interesting, but what i'm trying to communicate here is that there may be studies of this kind which never make it into a review article because they are of a more historical nature than an etiology-seeking nature, and therefore are not the kind that get wrapped into topical review

982:

Ok, I signed it, thanks for pointing that out. Apparently you have not understood MEDDATE: "If recent reviews do not mention an older primary source, the older source is dubious." That's pretty direct. Failure to understand does not mean that you are being stupid, and nobody said that you were, as

721:

biochemistry is not necessarily biomedical. the context matters. but everything in WP should be sourced to secondary sources - every policy and guideline urges that. secondary sources are one of the linchpins of NPOV/NOR/VERIFY. relying on secondary sources greatly reduces the chances of people

413:

in Japan, or this one about the levels of PCBs in people who lived in Brescia and worked in the factory that made PCBs, is it really a good idea to require the level of strictness that MEDRS confers? Is it really worth the tradeoff of having far, far less information about chemical spills, if there

2477:

This is very confusing. We usually accept the conclusions of review authors unless there is unequivocally something wrong pointed out by other RS. Your objections are not based on anything like that, but the uncomfortable fact that research from these countries which supports acupuncture cannot be

1886:

It would be very silly to pretend this issue does not exist. It would be equally silly to pretend that "Western" scientific sources have an equal and opposite bias. The problem of systemic bias applies especially to China and Russia, the literature in most countries has no systemic bias other than

1831:

A 2009 study by a Chinese computer scientist estimated that research fraud was a $ 150 million-a-year industry in China (in the form of ghostwriting services, fabrication of research, bribery to bypass peer review, and forgery). The study is mentioned in a number of the above sources, although I'm

1600:

It's not often that we get solid reviews proving consistent and widespread systemic bias in ONE particular subject area (TCM/acupuncture), and the researchers recommend that "Researchers undertaking systematic reviews should consider carefully how to manage data from these countries." We should do

1089:

The two are not independent. While secondary sources are sometimes acceptable beyond five years in areas where little new research is being done, primary sources are normally a short-term measure to be used for only enough time to allow secondary sources to be published: one or two years should be

401:

Please help me to understand the scope of MEDRS, because it is being used quite often to strike my edits which mostly concern harm done by chemical spills, at the moment, as well as dynamics about glyphosate and simple statement like the presence of a certain enzyme in microbes that are in the gut

1657:

There is no speculation. Some of the sources are used as references. The very idea for this content comes from referenced peer reviewed reviews on the subject. These countries have a well documented systemic bias using shoddy research to promote acupuncture and TCM. That's a huge red flag, and we

3106:

I think that "red flagged" is unclear; something like "should be used with caution" would make more sense. I'd drop the elderly Vickers source, and I'm not sure that peer-review scams is either accurate (it affected journals all over the world, not just in China) or necessarily worth mentioning

2019:

Ah! Okay then. I think we believe the same way. I just tend to see "bias" as a positive or negative leaning, and the negative being the same as a "prejudice", but that's just my careless use of language. I'm a bit "language confused", since I've lived in Europe most of my adult life and my daily

1981:

Yes (and actually yes to both, but I meant what I said). Science, as a body of knowledge or system of inquiry, does not care in advance what the outcome of a test will be. If a test refutes the null hypothesis and provides robust proof that homeopathy works, then science will have to pick up the

1300:

I'm not sure that this information should be included here at all, but I'm certain that this isn't the best way to handle it. It declares that almost all publications by nearly all Asian researchers are suspicious based on the actions of just a couple of individuals for the peer-review scams.

687:

the intro paragraph to MEDRS was written carefully. you will see that it mentions "health information", "biomedical information", "medical knowledge", " medical and health-related content". That was done on purpose to avoid wikilawyering efforts to work around it. It is content about health -

937:

As to your wording suggestions there, that's fine for me to hear and consider and discuss with other editors. I appreciate those distinctions. I wish it could be in a positive spirit of cooperation instead of an addendum to a fractured and contentious dialogue. I would love to be methodical and

858:

Jeez, your level of condescension in naming this edit "still not understanding?" As for "outdated" -- the research was done when it was done. As for the phrasing, that's up for suggestion and i'd take your point about the present tense being used. Sure, that makes sense. It could say "levels in

932:

care, very much, and that is why i am here asking for serious dialogue about this. I hve read and quoted the policy and asked for clarification, and yet i find that there is a smell of bias here. Here's an example. I understand the concept of "dated" and i do know that more recent research is

875:

Sage, you've been here about a month, and during that month you've been in conflict with nearly every editor you've had contact with. I suppose its possible that the dozen or so of us that you've interacted with are inappropriately censoring you, are unable to properly understand Knowledge's

325:

I agree with the viewpoint, expressed above, that impact factors have very little utility in evaluating the quality of a journal or individual papers. I guess I'm a little concerned, though, that the context seems to include a noticeable subtext of trying to shoehorn obscure, low-profile, or

646:

The content there does not just report measurements but makes reference to 'reference levels". I am not too happy with lumping "residents of some urban areas, former workers of the plant, and consumers of contaminated food," together but i am sick of dealing with this so did the minimum.

1577:

issue. The people who can correctly interpret and apply the advice you're trying to give them already know there are issues with some of this work, and the people who don't know it don't understand what they're reading anyway and will try to lawyer around whatever advice you give them.

1540:

I do think the narrower wording is better, though perhaps with an acknowledgement that the problem may be broader. (I would also be careful to distinguish this as very different from saying non-English sources are more likely to be biased. It's true that the highest-quality research is

496:

This is a very real case in which there are people who were exposed to PCBs by a factory, and i put a simple statement in the article on PCBs that levels were 10-20 times that of non-exposed populations in former factory workers. Is are you calling this a synthesis? I cited this

3166:

Li J, et al The quality of reports of randomized clinical trials on traditional Chinese medicine treatments: a systematic review of articles indexed in the China National Knowledge Infrastructure database from 2005 to 2012. BMC Complement Altern Med. 2014 Sep 26;14:362.

2802:

1381:

518:

Research on the adult population of Brescia showed that residents of some urban areas, former workers of the plant, and consumers of contaminated food, have PCB levels in their bodies that are in many cases 10-20 times higher than reference values in comparable general

887:

Sage, you've been given links to the policies and guidelines that pertain, yet you act as if you don't understand what they say. Either you don't understand, or you don't care. I'd prefer to believe the former, but you are making it difficult. "Dated" refers to

3519: 900:. The apparent absence of secondary sources, even after all these years, could be taken as implying that the primary source you want to use is not seen by experts as worthy of note. Now, is there something about that which is still unclear, and if so what? 170:

I would not use the Seneff paper to support any claims, however, because it's a loose and wild ride in possibilities without careful research into whether any particular claim is likely or even truly possible. It uses sources loosely, and it speculates

418:? I stand on the side of, the more information the better, about chemical contamination events. If it's used to simply support toxin level readings, and not so much interpreting about effects or anything else, then really, isn't it sufficient? Thanks. 688:

stuff people care about because it affects health. In other words, based on everything you have said about why you are here - exactly the stuff you want to write about the most. The effect of glyphosate on people, the effects of PCBs on people, etc.

672:

In this question, i'm seeking to understand the scope of what falls under MEDRS and why. What's thew spirit of MEDRS? I've read the page, of course, and in the talk page archives, i searched for "scope" and read a lot. Still, i find it under-defined.

1982:

pieces, rewrite the laws of thermodynamics, conservation of energy and mass action, and so on. The point about SCAM is that a positive outcome seems, according to the evidence, to depend on the investigator having a vested interest in the outcome.

3103:) situation, although strictly speaking, it ought to say "As of 2014" (publication date) or "As of 2012 (most recent sources used in the study). Also, saying "from China" identifies it as a geopolitical problem rather than a racial problem. 2478:

assumed to be reliable. That undermines the foundations of your support for acupuncture, which is understandably an uncomfortable position, but not grounds to reject a peer reviewed review based on your own OR speculation or own OR analysis.

3426:

Blinding ourselves to source bias because it's impolite isn't the answer. There's a noted publication and results bias favouring TCM from Chinese sources, which means that we need to tread carefully when using them. Similarly, we know that

387: 859:

former factory workers were found to be....". As for "still not understanding?" -- heck, i understand a lot of things, and this is a dialogue that we're supposedly having, to hash out the finer points of things. What's with the tone here?

3020:

I think we should at least include more sources in the reference, as well as the wording of the actual six factors, and also deemphasize "publication bias", since it is not the only factor covered under "systemic bias". Thanks so much to

2268:

DrChrissy, I am so sorry I got that wrong! You are absolutely correct! I changed my statement above to more accurately state what you said. Please feel free to correct me anytime I get something wrong like that. I can't apologize enough.

248:

has been cited more than 4,000 times, according to Google Scholar, but hopefully we're not presenting its conclusions as valid. There is ample evidence that retracted or discredited articles continue to be cited at an alarming rate (e.g.

3367:

I do not support the idea of mentioning specific cases in MEDRS-- have we mentioned the frequent copyright issues found in Indian journals? Where does it stop? Content areas can have FAQs, or we can make a general FAQ for this page.

2481:

By contrast, the rest of us are taking the review at face value as a call to increased awareness and caution when dealing with positive research, especially from these countries. This is a huge red flag which should be part of MEDRS. --

1844:

MastCell, I don't doubt it at all. Actually, that's why there are studies on the publication bias, and I think we should first preset those before jumping any conclusions of our own. Scientists are scientists, we are Knowledge editors.

2473:

of study due to personal objections to the study's inclusion criteria, references, funding sources, or conclusions." It seems you are pushing for rejecting the authors' conclusions if their statistics don't pass your peer review.

282:, and many editors have expressed similar opinions. Impact factor cannot be used alone determine whether the source is reliable or not, a common mistake that some editors regularly make. I think LesVegas earlier well pointed out: 1037: 3185:"Some countries publish unusually high proportions of positive results. Publication bias is a possible explanation. Researchers undertaking systematic reviews should consider carefully how to manage data from these countries." 2818:"Some countries publish unusually high proportions of positive results. Publication bias is a possible explanation. Researchers undertaking systematic reviews should consider carefully how to manage data from these countries." 2630:"Some countries publish unusually high proportions of positive results. Publication bias is a possible explanation. Researchers undertaking systematic reviews should consider carefully how to manage data from these countries." 1397:"Some countries publish unusually high proportions of positive results. Publication bias is a possible explanation. Researchers undertaking systematic reviews should consider carefully how to manage data from these countries." 1159:"Some countries publish unusually high proportions of positive results. Publication bias is a possible explanation. Researchers undertaking systematic reviews should consider carefully how to manage data from these countries." 1121: 947: 781: 427: 1337:

acupuncturists just fine. This deletion undermines our efforts to improve the quality of our sources by warning editors to avoid shoddy research. I'm sure it can be worded better, but it should not be removed entirely. --

876:

policies and guidelines, and/or are pro-industry shills, but given that you apparently came here after having similar disagreements on other websites, perhaps its time to consider whether your own behavior is the problem?

1924:

be against such notions. They are not founded in reality or promoted by RS. We will still document them, because they are part of the sum total of human knowledge and experience, but we won't give them any credence. --

2556:

So have we been too simplistic? The problem is larger by a factor of six. We have used the term "systemic bias", rather than "publication bias". Is "systemic bias" inclusive enough to encompass those six factors? --

3341:

Better. Shall we follow the modern approach of making the perfect be the enemy of the good, or shall we be all old-fashioned and wiki-like, and start here and improve later if someone has brilliant insights?

118:

and several other journals have low impact factors. However I am not sure if they are really so low that we should not be including review articles published in them as sources. What do other editors think?

2160:

Why not? I have requested numerous times for the studies on the publication bias. Whether it is there, or then it's not. If it is, please bring it to the attention of all of us. Thank you a lot in advance,

529:

health decisions badly in the future, or is it historical data? Is this a synthesis or is it ok that i am simply reporting readings of PCB levels that were in the paper? Please take my question seriously.

1601:

the same in our evaluation of sources. This is clearly an improvement of MEDRS and should be restored. (Keep in mind that the initiation of this thread is extremely misleading and not to be trusted.) --

2020:

language is Danish (or Swedish or Norwegian), rather than my mother tongue, which is English. That means my English grammar, punctuation, etc. aren't always correct, so don't hesitate to correct me.

3469:

for misuse, but shouldn't have any long-term effects. If TCM or ayurveda produces any results that will survive objective examination, researchers from other countries will eventually report them.—

1754:??? The references are right in the content. Also they are presented below, with more, by others. The bias is well documented in high quality sources, and it's a huge bias we should not ignore. -- 1025:

primary sources, then you read what I wrote, decide that it's outdated because of a study published after my review was written, and so you publish your own review, to include the newer source(s).)

1627:

That indeed might be the case, but instead of mere speculation we need sources to confirm this. Publication bias is quite regularly studied, it shouldn't be a problem to find such study. Cheers!

2571:

No problem, apology accepted. It's good we got the misunderstanding solved out now. Anyway, I think we need find a good way to address the problem with respect to the sources in whole. Cheers!

194:

Low impact factor depends on the subject--there is no general rule. It is generally more useful to look at the impact of individual papers, which is what impact factor is designed to measure.

3069: 2580: 2566: 2543: 2141: 2116: 1868:. If that doesn't scream systematic bias I don't know what does. That level of bias I'm sure has had the intended effect on skewing systematic reviews in favor of their chosen result.... 1763: 1667: 1636: 1610: 299: 1968: 1536:

as well. I also wouldn't characterize the problems described in citation 4 as "a couple of individuals" - they do provide an example but use it as an illustration of broader issues, e.g.

3536:

That makes it easier since it already exists. :-) I already started drafting something, so I'll paste in some of the stuff I have and will leave the incomplete parts in hidden comments.

2332: 998:

We can't assess these things, we must leave it up to published experts. We need those secondary sources. Please accept this and direct your considerable energy to finding those sources.

320: 3499: 3131: 2278: 2090: 1887:

towards reality and empirical verifiability. The idea that "reductionist science" is biased against empirically unverifiable notions is a fallacy to which Knowledge should not succumb.

1854: 1538:"China's science ministry commissioned a survey of researchers...one-third of more than 6,000 surveyed across six top institutions admitted to plagiarism, falsification or fabrication." 153: 3416: 1587: 1346: 1327: 1101: 1084: 1055: 3091: 2428: 2410: 2393: 2378: 2346: 2323: 2296: 2263: 2195: 2174: 2155: 3531: 3513: 2200:

May I respectfully correct LesVegas, I believe I only asked the question of whether it was racist...I don't think I made an unequivocal statement that it, or the editor, were racist.

2081:". Here, "might be" might have two meanings: whether it has not been studied (quite basics), or more likely it has been studied but it didn't yield any results worthy of publishing. 1877: 1749: 1708: 3402: 2510:

and Taiwan) in general. It suggests that publication bias is a possibility among five other possible explanations, but it does not study it. The other five explanations are 1) the "

97: 89: 84: 72: 67: 59: 3551: 3351: 837: 808: 183: 3478: 3459: 1009: 911: 880: 868: 853: 589: 572: 538: 469: 1998: 332: 213:, is there an easy way to do that? I think some of our editors are using impact factors for journals because it's an easy way to provide an "objective" objection to a journal. 3440: 2305: 2213: 2047: 1934: 379: 360: 239: 222: 205: 130: 2360: 977: 611: 1564: 739: 712: 697: 682: 656: 1781:← There are plenty of reliable sources to substantiate the concern over research fraud in China, often attributed to the academic culture there. A short list would include: 158:

I would support this notion. Cut off the absolute low ratings, and then take it on a case-by-case basis with the impact factor having some impact on the overall discernment.

1900: 267: 137: 842:

You are still trying to use outdated primary sources, and worse you are using them for statements that are both in the present tense and in the voice of the encyclopedia.

730:. (the original primary source was retracted in that case... it is not always that severe, but lots of primary sources in the biology space turn out to be nonreplicable) 625: 616:

Yes, i very much see that danger, Guy, and this is why i was asking specifically about measurements relating to historical incidents, and not interpretations of etiology.

2283:

yes. you should not discuss sources you haven't read. that is scholarship 101. if your scholarship is so abysmal that you actually need to be taught that, please read

2351:

Absolutely in agreement, Doc! I would hat them off myself if I knew how, so by all means go ahead. The statistical discussion above all of that is far too important.

2284: 288:

8-10 are a good benchmark for biosciences. But the American Journal of Sports Medicine (the highest ranked orthopedic journal) isn't even close to that high! (4.699).

2606:

There are serious red flags regarding Chinese medical research. Especially China, Japan, Russia/USSR, and Taiwan have been shown guilty of systemic bias in favor of

1135:

There are serious red flags regarding Chinese medical research. Especially China, Japan, Russia/USSR, and Taiwan have been shown guilty of systemic bias in favor of

114:

regarding whether certain sources are reliable, and one issue he has discussed is the impact factor of the journals in which they are published. He asserts that the

524:

Suppose that there is a single editor among 10 who appears to want to minimize any statements that may sound bad for chemical companies. Say that this editor cited

3382: 3116: 3034: 2491: 1333:

is far from "a couple". If we find such reviews showing systemic bias in other countries, we can add that. There is no reason to reject what we do know to be true.

1317: 630:

the point of MEDRS is to have reliable content about health. Toxicology is part of health and everything that Guy said applies to that as well. With regards to

502:

Turrio-Baldassarri, Luigi, et al. PCDD/F and PCB in human serum of differently exposed population groups of an Italian city. Chemosphere 73.1 (2008): S228-S234.

174:

There may be valid claims in the paper, but they are not developed with a degree of certainty that warrants serious reporting, yet. It is a good paper, though.

2071: 1838: 824:

completely reasonable as a solid sentence about this contamination site, and so i had to come here for some interpretation, because i don't think this is what

47: 17: 3335: 458:

from primary sources. Feel free though to create that synthesis and get it published elsewhere, in a reputable peer reviewed journal. Then it will be usable.

1526: 3316:

Dropped the reference to peer-review scams. I replaced "peer-review system" with "academic system" since that's better supported by the remaining reference.

2314:. Back to statistics, is there any studies on the publication bias, or is it mere speculation? I am also waiting for the source to check it myself. Cheers! 2252:

Do you have an access to this article? If so, could you please send me that one? I'd like to see if and how they might have possibly studied the subject.

3388:

I do not support the inclusion of MEDRS additions like this. At best, I agree with SandyGeorgia above, but in that instance, for different reasons. As

3357:

If we are still (at this point) talking about adding something to MEDRS to deal with specific issues, articles, or sources, I think Zad gets it right:

2548:

Excellent response! Respect and great thanks, and apologies for the impatience, coming your way. We simply didn't understand what you were driving at.

1821: 351:

that might be useful to editors and maybe even save you a bit of time eventually (if it means that you won't have to repeat yourself over and over)?

2595:

To make this easier, I'm starting a new section. Here are the three suggested wordings so far. The first two use the 2014 Vickers review, and more:

2940: 2254:" No need for facepalms, just try to keep cool, will you? So far, the conclusion doesn't sound really confident about it's own findings. Cheers! 1028:

If the source he cited was four or five years old, rather than seven, would you still be objecting to its inclusion (specifically) per MEDDATE?

2132:, so that would indicate that it hasn't been studied in that particular report. Anyway, that's not WP:OR, BullRangifer, that's a direct quote. 703:

the review? Like, for example, the three subpopulations who have 10-20 times the PCB levels of baseline populations of some congeners of PCBs?

791:

a statement like "In 1932, one study reported that the level of X in Italy was Y", but if you need to use that source, is that detail really

2401:

i emailed you through the WP system at 11 AM and have nothing back from you. I guess you are getting the Vickers reference some other way.

485:

is that it's a special-case stricter sourcing guideline to make information that may be used for health claims , which means it's more than

1869: 1812: 637: 631: 313: 3407:

No support for using specifics in MEDRS per SandyGeorgia. Sources are reliable or not per the specific content they support. Simple.(

2248: 283: 598:

Including primary sources leaves us prey to repeating the beliefs of the lone maverick, before they have been analysed or rebutted.

1323:

Yow! And the assertion is that English language publications are less biased? Don't think we want to get into that food fight.

1532:

This is a major issue which does exist, not just in China, though it is probably best documented there - e.g. I'm also aware of

2328:

If LesVegas is in agreement, I am happy to have the above 3 comments and this one hatted as "Off Topic" or something like that.

787:

I think that a good primary source (=peer-reviewed/decent journal/not contradicted or disputed by other sources) can, indeed,

2614:. Editors should consider carefully how to manage sources from these countries. The problem also includes peer review scams. 1143:. Editors should consider carefully how to manage sources from these countries. The problem also includes peer review scams. 254: 1021:

What's the approximate review cycle length for this specific subject? (Review cycle = I publish a review article based on

111: 105: 3428: 2419:! I'll have a look tomorrow, it's getting rather late here right now ... although I have a day off tomorrow! :-D Thanks! 393: 2031:. Prejudice implies a preformed judgment even more unreasoning than bias, and usually implies an unfavorable opinion: 3017:

at the end of the previous section? Does "systemic bias" still cover the subject well enough, or should we add more?

957:

So, in summary, i would like admission or denial, with cause, as to the fact that the claim is not an etiology claim.

643: 2077:

Exactly. I am still waiting for those studies on the publication bias. I don't want to hear any explanations like "

727: 723: 2499:

That undermines the foundations of your support for acupuncture, which is understandably an uncomfortable position

2079:

Some countries publish unusually high proportions of positive results. Publication bias is a possible explanation.

2590: 1916:

I understand what you mean, but maybe this is what you intended to say...? "The idea that "reductionist science"

1533: 1017:

I don't have time to sort through this dispute personally, so I'd like some quick answers from the objectors:

3150:. Such sources should be used with caution. The problem also includes issues with the academic system in China. 1308:

The English Knowledge has enough of a problem with systemic bias. I don't think that we should encourage it.

115: 2953:

Chinese authors use more Chinese studies, which have been demonstrated to be uniformly positive in respect to

1545:

of science - e.g. the language of the top journals - but that is a separate issue which is unrelated to bias

3147: 2783: 2607: 1362: 1136: 1862:

found that 99.8% of randomized controlled trials of acupuncture published in China had a positive result.

1859: 2786:. Such sources may be red flagged. The problem also includes issues with the peer review system in China. 1365:. Such sources may be red flagged. The problem also includes issues with the peer review system in China. 161:

The Seneff paper on glyphosate was published in the journal Entropy, which has an impact rating of 1.564.

3099:

I particularly like some aspects Jytdog's phrasing: "As of 2015" identifies that it is a current (but

3065: 2576: 2539: 2424: 2374: 2319: 2259: 2170: 2137: 2086: 1850: 1745: 1632: 1583: 544: 295: 38: 3455: 3412: 2859: 2678: 1438: 1207: 415: 3289: 3211: 2926: 2885: 2844: 2759: 2711: 2663: 1964: 1505: 1464: 1423: 1288: 1240: 1192: 348: 2520:

judgements of whether the test treatment was superior to control were, in some cases, subjective.

2506: 1126: 1095: 1049: 1003: 905: 847: 566: 463: 278:

I find it easy to concur with Kingofaces43. There's also been discussion about impact factor at

3527: 3495: 3378: 3347: 3127: 3112: 3030: 2562: 2487: 2112: 2043: 1930: 1759: 1663: 1606: 1342: 1313: 1080: 1033: 804: 356: 218: 149: 125: 3219:

Ernst, Edzard (2012). "Acupuncture: What Does the Most Reliable Evidence Tell Us? An Update".

2974:

Ernst, Edzard (2012). "Acupuncture: What Does the Most Reliable Evidence Tell Us? An Update".

963:

Thirdly, in light of the second claim, the preference for more recent research is less strong.

3389: 3190: