1

Background :

While reviewing Very Low Quality, I noticed that some answers which are answering questions or answering partially. What they lack is just a reference. This meta is about discussion regarding giving time for improving answers as stated in so called Official policy for deleting answers that don't cite sources.

We know that our site has a policy which mandates some kind of reference in our answers. Not everyone can go everywhere with books. Some are broad sources like the chapter names or a part from Puranas. User know that the answer will be present in a specified chapter. But we are not getting early answers. As a result, we are getting series of long comments which is not helping the site in any way. Just the OP. In this case we have to encourage users in posting answers explaining first and then the answer can be improved anytime.

That is the case for established users. But we have a total different case for new users. New users do not know that we have a policy which mandates some reference. But they post answer anyways. Some are really good answers. They answer the question anyway. Who knows, that answer could turn out to be a good one and that user shall become a regular user who will contribute regularly for the site. We need to give them guidance and support how to improve their answers and give the choice of writing. What they lack is the direction how to improve their and the source. We have brilliant guidelines for new users, some good resources to help the user to find information to cite and many great meta discussions so that the user will gain interest in contributing to the site.

But.. that is hardly happening. Unsourced answers are flagged very soon in 1 or 2 days. Average time they are getting flagged is not exceeding 2 or 3 days

According to Official policy for deleting answers that don't cite sources, the time given for users for adding sources is a week. For new users, it is a fortnight i.e., 2 weeks. That is agreed by almost all (I disagreed because the time span is less. anyways,). According to that policy, We should give time to the OP for improving their own answers. If not, we can help the OP by directing them to valid sources which can be cited. If said properly, users will understand which area in the answer needs improvement and try to modify.

On the contrary, answers which lack sources are either flagged as Very Low Quality or Not An Answer within a day or sometimes in an hour without making any efforts for their improvement or guidance Just a canned comment to cite sources. I wrote about this practice in fourth anniversary meta post in my answer.

Lately, I have noticed that answers which have no references are flagged as Very Low Quality (VLQ) very quickly i.e., less than 30 minutes after posting. This is a very quick move to flag an answer as VLQ just because it lacks sources.

The flag has totally a different purpose. It should be used

  • when the answer has poor formatting which can't be improved by editing

  • when the answer is written in Non English language and gibberish language hard to understand

  • It asks a new question or a comment

  • A site specific policy like we have personal experiences. It can be used when the answer has personal experiences as sources.

Again, formatting can also be improved if time is spent. What guarantees immediate deletion is spam, inflammatory speech, blatant non answers, comments and answers posted as a separate answers. Unsourced answers do not guarantee deletion immediately. As stated above, we even have a policy which moderators themselves have forgotten. What to speak of normal users?! Lately, I noticed many answers without a source are being flagged as Very Low quality or Not an answer. There was a 2 day old answer flagged as VLQ or NAA. What is surprising is the answer was fine on its own merit. It answered the question precisely. The reference was present in indirect speech. Only thing it was missing is a number where the verse is said. Even then, the answer is flagged as Not an answer or Very Low Quality (which is to be used when the answer nowhere addresses the question or it is a comment.), moderator commented to cite sources and another moderator added quotes himself and the answer was reviewed as "Looks OK".

Even though I agree with adding sources to help the user, this will not turn out to be a good practice in a long run. When we are doing anything to an answer, we should let them know what is happening to the post. We have to encourage users to cite their own sources and engage them and it is the best if the input comes from them. If they don't respond in the prescribed time, we can add sources if the answer is salvageable. If not, it can be deleted. But intervening to flag, editing on our wish etc., will look condescending which no user really want to see. If the user returns after a week or two just to find their answer deleted with no explanation except a canned response :

"Answers should be backed up with sources on the site. Add sources and flag for undeletion".

If this is seen every time without really explaining why the answer should be modified, how answer can be modified, the experience is not a good one for a user. They can't comment on their own answer if it's deleted. Only some users know there is a meta and chat. Even then chat requires some reputation to talk. So, keeping in these mind, it is better to explain which portion needs sources, whether we have added or the OP in comments is always helpful. It is not possible for many users can't find the required sources immediately given that we get questions on wide range of scriptures. Many need to be verified and added. So, giving some time is better.

Commenting and putting banner as step by step has given some good response. In my activity here on the site, I saw many users responding to reviewer's comments under answers either by editing something in their answer or explaining why their answers are good. Any reply from OP is okay for us. If there is a reply, we can try to improve and make them understand our site's policies. We have to remember that every user here joined a site as a newbie and we can't expect all to have full understanding of SE and our site specific policies. We should comment or moderate in good faith that users would return and respond to our comments and NOT thinking they would never return.

If there is no response from other side, we can continue our moderation activities like flagging and deletion. But what is the hurry of flagging or editing directly without engaging the OP in the conversation?

So, should I let the unsourced and low quality stuff hang around?

No, not at all. We have to remember that flagging should be the last resort. Depending on moderators for every thing is not a good way of moderation for a site like Stack Exchange. We have to use our votes wisely. Even when there are answers flagged as Very Low quality, I am not seeing any downvote or a feedback comment on the answers. If an answer deserves deletion, why does it not deserve a downvote? Why are users not willing to downvote and letting the OP know that there is something which can be improved? So, using downvotes, comments and other mechanism we have on the site to let the user know that their answer has issues is the right choice before flagging as Not An Answer or Very Low Quality. Not every unsourced answer is a non answer.

If the OP is inactive and there is no response from the other and the answer can't be improved in any way, we can let the OP know about that and flag answer for moderator intervention and ask for deletion. Hence, we still have a way to get rid of those even after two weeks.

So, what I humbly ask the community and the moderators is

  • Be patient while flagging unsourced answers.

  • Let the user know how to improve their answer on their own and give required guidance in comments. Many will respond with edit or comment if they are active.

  • Guide them towards helpful meta discusisons and resources posts we have on our main site.

  • Use your votes and privileges wisely. Votes can be reversed once the answer is improved. (This has happened many times with me and the result was positive and I got comments thanking for guidance.)

  • Wait for the time span to end and then intervene in flagging or editing drastically with the sources we know. In the mean time, it is very much needed to keep the OP engaging so that they will know how to write answers and we will know which type of sources they are looking for.

  • Do not post answers in comments. Post answers as early as possible. They can be improved anytime later. It helps our answer percentage.

This is what I propose based on previous meta discussions and . I'm welcome to suggestions from community on how to improve moderation of unsourced answers from new users.

Note: This discussion is only about unsourced answers which answer the answer and has some content which can be improved by community effort. This is not about comment replies to questions, Thank you comments, new questions, personal opinions, rants posted as answers. They can be flagged as usual for deletion.

4
  • 3
    Do you have any data on how many new users actually respond to comments asking for sources and improve their answers? That would tell if waiting for 2 days, 2 weeks or 2 months makes any difference or not. Commented Jun 27, 2019 at 20:15
  • 2
    @sv. I have personally seen response from the OP on the answers which are first flagged as VLQ and there is no single downvote or a comment. Later there were comments that the answer contains objectionable content and should be reworded. This made the OP to modify or respond through a comment .This happened a couple of days ago. So, I am positive about commenting first and then edit or delete. If there's no reply, we can continue our moderation process. We shouldn't be sure that he never returns. I just quoted that our faq that we should give a week before deleting. Commented Jun 28, 2019 at 8:18
  • "we even have a policy which moderators themselves have forgotten." How have we forgot the policy?
    – Pandya Mod
    Commented Oct 22, 2019 at 14:45
  • 1
    @Paṇḍyā Because moderators were deleting unsourced answers before the prescribed time if they see in the review queue. If not deleted, some action in between is taken like a direct banner without comment or editing with own sources even though sources were already present. I gave one example in the question. If the review history is checked, accurate details will be known. Everything is related to official policy of citing sources and its implementation. Commented Oct 22, 2019 at 16:10

2 Answers 2

-1

I welcome suggestions from community on how to improve moderation of unsourced answers from new users.

Other than a few anecdotal accounts, there are no real stats available on how leaving comments asking users to back up their answers is really helping this community.

Assuming most new users don't respond to such comments or update their answers (see examples below), here are my suggestions:

  1. Since answers without references are very common from new users of this site, instead of expecting reviewers to raise a custom flag and explaining the obvious to mods over and over, we should have mods interpreting VLQ/NAA flags as lacking in sources and either add a citation notice or delete the answer if it looks more like a comment.

  2. If someone feels strongly that an answer which can be improved by finding and adding the right sources has been deleted due to VLQ/NAA reviews, they should flag the mods to undelete it and add a citation notice.

  3. When the site graduates, we can request SE Inc. for a feature where users can directly flag and add a citation banner/notice without mod intervention. The banner will contain a link to this meta post. Until such a feature is available, instead of interpreting VLQ and NAA flags literally and rejecting them, mods should try to see what's wrong with the answer and leave appropriate notices and comments.

I think following the above will greatly reduce duplicate effort and everyone's time when dealing with unsourced answers.


Example answers from 2019 where users haven't responded to comments asking for sources:


A few from 2018:

14
  • We moderate keeping in mind that new users are going to read our comments and feedback. Let it be policy making, editing, commenting etc., If we are not waiting for other party to know what we are doing, it will lead to one sided participation. It will only be fruitful if we are including everyone in our moderation. We should do our part of informing and waiting. If they are responding and listening to us, that's a desired result. If there's no response, we will continue moderation anyway. What's the problem in waiting a week? Commented Oct 23, 2019 at 16:19
  • 'What's the problem in waiting a week?' - we wait for a week then someone upvotes an answer with zero references. Then you cannot flag as low quality and are forced to raise a custom flag next time you see it. Ideally, one should flag the mods to add a citation banner first time they see a LQ answer but I want to avoid custom flags as much as possible. We should not force users to use custom flags for the most common problem with answers on this site. It should be a simple click and mods should take care of the rest. @Sarvabhouma Commented Oct 23, 2019 at 21:55
  • If mods keep rejecting VLQ/NAN flags on LQ answers over technicality I may stop flagging them altogether @Sarvabhouma Commented Oct 23, 2019 at 21:56
  • "We should not force users to use custom flags for the most common problem with answers on this site." Yes, that's true. Moderation should be in low level as possible. These notices also need moderator's intervention. I personally haven't seen much usefulness of these notices. We should be using normal flags and last resort should be a custom flag. As you have mentioned, there is a problem of sympathy voting on the site. I don't have any ideas right now to deal with it. Seems like every beta have seen such issue. Commented Oct 24, 2019 at 1:49
  • Flagging unsourced answers as VLQ or NAA is wrong. VLQ and NAA are not for that purpose.
    – Pandya Mod
    Commented Oct 24, 2019 at 2:35
  • @Paṇḍyā I consider unsourced answers as VLQ on this site. We flag and delete unsourced answers on this site which means they don't meet the criteria for a good answer on this site. When an unsourced answer is upvoted and cannot be flagged VLQ the next best option is to flag it as NAA - I agree it doesn't fit the NAA definition but you can't expect reviewers to flag mods with custom reasons when there are a ton of unsourced answers on the site. As I explained in the answer there should be a "No References" flag on this site which will automatically add the citation banner. Commented Oct 24, 2019 at 20:59
  • @Sarvabhouma 'I personally haven't seen much usefulness of these notices.' - I find them useful because they serve as a warning to readers and also as a reminder to OP to update it with some references. If you have to choose between deleting an unsourced answer vs. keeping it in some form better to keep it with this notice. Commented Oct 24, 2019 at 21:03
  • @sv. "I consider unsourced answers as VLQ.." No, that is wrong. Read "Let me address possible confusion..." paragraph in my answer.
    – Pandya Mod
    Commented Oct 25, 2019 at 0:37
  • @Paṇḍyā I don't agree with your reasoning nor OP's which is why I wrote a separate answer. You are going by literal interpretation and semantics of the flags. This site has a separate standard of quality than other sites. Let's not go by how those flags are used on other sites like SO where personal experience is a valid source but here it's not. How can you salvage an answer like this one with a 100 lines of unsourced statements unless you are willing to spend 30 mins? In 30 mins. I can write my own answer instead of fixing someone else's answer! Commented Oct 25, 2019 at 1:30
  • @Paṇḍyā "'I consider unsourced answers as VLQ..' No, that is wrong." - It is low quality according to this site's standards. As a mod if you ever deleted an unsourced answer you are agreeing in principle that the answer has no place on this site. Give it whatever name you want, 'low-quality', 'very low-quality', 'great answer but lacking sources', but at end of the day it will be deleted, in a few days or after 2 weeks. Commented Oct 25, 2019 at 1:37
  • @sv. No, that's wrong concept. Stack Exchange doesn't detect our source quality when they mean VLQ.
    – Pandya Mod
    Commented Oct 25, 2019 at 1:44
  • 'Stack Exchange doesn't detect our source quality when they mean VLQ' - Of course it doesn't so we tell SE what is VLQ and what is not. That is why we have flags. Different sites have different criteria for questions and answers. Again the community decides what is a good answer for a site. If this site didn't have the 'cite sources' rule, then we don't have a problem to begin with. Without offering any real solutions you are simply saying 'using VLQ /NAA is wrong; flags will be rejected'. @Paṇḍyā Commented Oct 25, 2019 at 1:53
  • @sv. It's a suggestion from Stack Exchange (CM) not to use VLQ for addressing unsourced answer.
    – Pandya Mod
    Commented Oct 25, 2019 at 1:58
  • @Paṇḍyā What reason did the CM offer for not using VLQ? You should probably tell them to write an answer here? Custom-flagging unsourced answers will not work in the long run, IMO. Commented Oct 25, 2019 at 21:52
-2

You're mixing many things in the body of the question. If we talk about the problem stated in the title of question, the solution is:

VLQ queue is not for reviewing the unsourced answers.

Unsourced answers should not be flaaged as "Very Low Quality" or "Not an Answer"

  • VLQ flag is for gibberish contents which are not salvageable by editing and needs quick removal as soon as possible to clean the site.

  • Not an Answer should only be used when the answer doesn't attempt to answer the question.

Let me address possible confusion for the term quality that I think some users may have: when we say that citing sources does improve the post quality and in order to maintain quality standards we demand citing of sources while answering questions, the term quality stands for different meaning than that of indicated by VLQ flag reason. The term quality in VLQ stands for gibberish and unsalvageable content that needs removal whereas the by quality of sourced answer, we mean reliability and authenticity of post.

When you find unsourced answers, the proper action is to flag them using custom flag reason, called "in need of moderator intervention" explaining in the text box that answer doesn't cite any source. Moderators will put appropriate post notice. Do not flag them as VLQ or NAA. If this practice is followed, unsourced answers will not go to VLQ review queue and we will not face such an issue mentioned in the question.

"Not An Answer" flags for unsourced answers will be rejected and repetition of incorrect flagging and rejection of considerable flags will lead to ban from flagging as mentioned in the last point in Flagging section at Complete Rate Limiting Guidelines


7
  • 1
    What action will be taken on the users who abuse the privilege of flagging unsourced answers as very low quaity and reviewers selecting them to delete? What action will be taken on them? Also, this is not just about a few answers. They are flagged immediately or after 1 or 2 days forgetting policy post to wait for a week or 14 days. If the info about policy and asking users to wait for a week is also added, answer will be completed. Commented Oct 18, 2019 at 15:39
  • @Sarvabhouma Those flags will be rejected and representation of incorrect flagging and rejection of flag will lead to get banned from flagging as mentioned in the last point in Flagging section at Complete Rate Limiting Guidelines
    – Pandya Mod
    Commented Oct 18, 2019 at 15:48
  • I wanted this information to be added in the answer. Can you add this in the answer with a better formatting please? Commented Oct 18, 2019 at 15:58
  • It's already present. Why is it necessary to add this in the question again? If I add that, it will look like question to answer plainly and not something what you have added as a part of the discussion. Till now most part of the answer is already present in the question. I already stated "suggestions from community on how to improve moderation of unsourced answers from new users." I dedicated a whole paragraph in the question for reviews and flagging also. I think it's your part to add it on your own as part of your answer.. Till now, most of your answer is just repeating my question. Commented Oct 19, 2019 at 3:20
  • 1
    So, only point to point answer and nothing else? There was no need to add addressing the comment. I already raised about flagging and reviewing in the question itself. Since you insisted again to comment, I wrote a comment under your answer. Commented Oct 19, 2019 at 7:46
  • @Sarvabhouma check the revised answer.
    – Pandya Mod
    Commented Oct 22, 2019 at 14:44
  • 1
    How am I mixing many things? I think title alone is not the entire question. This is a discussion and there is more depth in the question to what led to the current discussion. There is immediate cause for any problem. That immediate cause I added in the question and how the policy is not followed. There are many reasons why I posted this discussion. If I am mixing many things, state them in comments. Commented Oct 22, 2019 at 16:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .