Originally published at: http://blog.tindie.com/2014/07/new-reviews-seller-policy/
Tindie turned 2 last month. With over 12,000 orders, 2,500 products, and 500 sellers, we’ve learned a tremendous amount during that time. We are in a unique time in history where people are making exciting, innovative projects, and are able to bring them to market faster than before. As manufacturing gets easier and cheaper, this will only…
It’ll be interesting to see how the ‘4 star average’ threshold plays out. Lower volume sellers with very few reviews versus higher volume sellers with more reviews, etc.
Interestingly enough, we’ve never seem to have any problems with lower volume sellers. It seems always to be when higher volumes pop up is where the problems arise. As everyone knows larger production runs are more difficult than small runs- and that seems to be reflected in the reviews/feedback we get back.
I’m certainly on board with the spirit of this. And as a low-volume seller I guess I’m an example of what Emile says, I’ve not had an issue. But the way the policy reads sounds to me like it will be (or could be) automated and that brings a risk of abuse. It sounds like someone could easily cause problems for a seller, maybe even another seller. Hope this has been thought through thoroughly. Some level of human oversight is needed. That’s extra work for the staff, but I think if the expectation is that it’ll run by itself, hands-off, then you will be disappointed and will find yourself in the middle of acrimonious arguments regardless.
“No exceptions” does seem a bit short sighted, TBH. A Tindie seller with an illness could lead to a string of bad ratings for ‘shipping’ and ‘communication’, and permanent excommunication without first attempting to resolve the issue seems an extreme solution. Perhaps a “performance plan” for people whose ratings are slipping would make more sense as a first step?
Also, less than four stars over what period? If it’s all time, a well rated seller could go a long time with poor service before their average dropped. If it’s a fixed period, a low volume seller could be evicted for a single bad review.
Hmm… the 4/5 star thing is a bit concerning. Certainly having good quality sellers and products is a good thing, but isn’t that “score” a bit ambitious?
3 may be a “good” rating to some raters as opposed to 5 which could equate “You kissed my ass” rating.
Or maybe someone is rating according to what they felt they got out of the project. So if someone said, well, I got a product and it technically does what it says it’ll do, but wasn’t given the code (because you know, not everything is open source), so I’m going to give the person a 1.
I think first seeing how the rating system does first before instituting some sort of penalizing mechanism would probably make more sense than automatically kicking people out…Esp to see if it hurts smaller sellers.
My 2c (and you know we love you and Tindie!)
Implementing something this drastic without a baseline of review data to go against, and with no opportunity for remediation is a bit shortsighted. You’re also basing this on a rather subjective system. I’m not too keen on this at all.
@arachnidlabs good points- also how to handle a new seller with only one review?
This will definitely evolve as we flush out the finer points (so no exceptions was probably the wrong choice of words). The main goal was to set a benchmark for selling on Tindie - and making it known that poor sellers aren’t going to be sticking around. Prior to today, we didn’t have this as a policy so to set the wheels in motion, need to set a line in the sand.
So I agree much to flush out, but we now have data to measure & a starting line.
To you finer points-
- Reviews over X days, sounds good, but what about lower volume sellers, so maybe over X reviews? That would work for new and old sellers alike
- Illness/Vacation etc are scenarios we still haven’t addressed. I think we can still play those as we see them (very small minority of cases)
I have never felt star review systems like this are very good, as another commenter pointed out it is very subjective.
If the service is adequate and everything goes to plan that could be a 3 star average review to some.
I believe this is why youtube changed to a simple good or bad system.
It is a bit aggressive, but I think it will be no problem for the vast majority of sellers. And this goes along with @tautic’s point about seeing the data - definitely going to be looking at the data over the next month (part of the reason we made it the 15th). We’ll see how the data looks and maybe we need to tweak it - but we have plenty of time. However we had to start somewhere.
To be frank, everyone that has replied to this thread aren’t the ones that I think this will impact. I think everyone here are the great sellers - it is the ones who don’t respond to messages and get mixed reviews which ultimately hurts everyone here.
Removing the bad apples will ultimately be better for everyone, and so we wanted to create a transparent policy. We at least have that and can tweak it as we get data back and see what it shows.
Any purely mathematical system breaks the instant humans provide unconstrained input to it. That means that only another human with knowledge of all the background can realistically judge if stars given to the seller were fair or not. I do understand that requires somebody going over bad seller ratings and making decisions, but in my opinion that is the only way if you want to keep the game fair.
1 Like
Hello there,
I am all for filtering and upgrading the quality of seller on Tindie!
I also can see where you’re coming from regarding how buyers view the site.
But, as a low-volume seller, I can say I have my doubts about the system,
as I am sure many seller have.
It should be more anthropocentric.
For instance seller can make dispute / complaints that get reviewed
and after X number of valid complaints the seller is shown the door.
Also Emile made a very valid point “what about sellers with 1-2 reviews?”.
And there is one question I must ask, will the star review by mandatory for the buyer,
or will we end up begging / nagging buyers to leave us 5-stars, like Eastern eBay shops do all the time?
(aslo, when will this be implemented?)
Thank you
Bill
Hey Bill,
Thanks for the reply. The reviews are already being collected right now. The date we announced yesterday will be August 15th for evaluating sellers. (Again I don’t see this impacting any sellers in this thread.)
There definitely are still some areas to polish, and like you and @msrl said, yes there must be some human decision. It won’t be automated. We definitely should have made that more clear, and not said “no exceptions.”
Will add summary post in next reply…
Emile
Thank you to everyone in this thread for adding your thoughts. It has been great feedback and I definitely agree with the vast majority.
To summarize:
- It won’t be automated
- Without a public policy & proper review system, it wouldn’t have been right to just cherry pick the ones we deemed as “bad”
- We wanted to set a framework and policies in place that were transparent and allowed us room to remove the ‘bad sellers’
- We don’t have enough data right now to tell if 4 stars is too high or low
Areas for refinement:
- Evaluate over an average taken from the last N reviews (not all time or over a timeframe)
- How many reviews should this be?
- A benefit of this - it will give us room for new sellers to get used to Tindie and build their reputation over time
I think that is a pretty good summary of the points so far. How does that sound?
If you want to “remove” a seller, then perhaps the best way is to establish a “Tindie Code of Conduct for Sellers”. The code of conduct outlines the basis for participation and then if a seller defaults on the Tindie ethics for e.g. quality and delivery there is a review process after which a seller is blacklisted, and then later removed.
My concern is that there is no way to control how a market or individual will respond to your products and there are too many things out of your control. For example I worked as a technical writer for some time and the best written documentation for technical products often got a “This is too hard, is there a 1 pager” kind of response. Product delivery to customers from countries external to the US and EU are at the mercy of shipping companies and their logistics etc.
We also need to keep in mind that considering the infrastructure most sellers have in place to deal with product returns, if a customer is throwing their toys because they want a refund because they are at fault for the purchase or misused the product should that really be the torture stake of the seller?
As much as we all want to deliver the best service at marketable prices, there is a line in the sand somewhere between where what the seller is liable for differentiates from the rights and duties of the buyer . . .
Hello Emile,
Thank you for the reply,
you, and everyone in Tindie, always help to clarify things in the forums.
This review system is a step in the right direction,
lets see how it plays out…
To throw in a curve ball - here’s another idea which I think will get more seller support-
We should have a seller agreement that outlines what is expected:
- Ship within X days
- Reply to messages within X time
Off of that, when a seller takes too long to ship or doesn’t reply, we notify them that their products will show up lower in search/browse and take that into account on how we display products on the site. Right now our sort is based off sales over time, but we could add these other attributes into that score.
Repeat offenders then we can handle individually.
So instead of so much weight on reviews only, we incorporate the seller’s actions. Thoughts?
I think you then have to then allow the seller to say how many days it takes to ship. I can sometimes take 2 days due to certain reasons. In the future it may be where I batch up my sales and ship every Friday (Ready to arrive for Saturday or Sunday [Yes there may be post on Sundays soon here!]) which isn’t just a benefit to me but to my customers as going 1st class means they will 95% of the time get it on the Saturday ready for hacking.
This is quite common in big companies. A lot of Free / Cheap PNP rates are delayed until 6 days after and sent next days just so they can charge more for next day.
The reviews system could also affect. If you only sell 1 and have a bad review then it shouldn’t be penalised at all. Maybe a percentage of their orders after 2? 50% of the sellers reviews are bad then they get lower down?
And most importantly
As I learned in Business studies a bad review is more likely to spread and even be given. Reviews are as much a way to get an issue resolved (and I do it myself because if not I don’t get the issue resolved). A good review is less likely to be given from where a person doesn’t see the need to.
I have only had 5 reviews on my motor kits via tindie. Depending I have had 156 sales this makes it a disappointing 3.2% What benefit does the customer have to leave a good review? Is the new system going to be easier and quicker for the customer?
I see your asking for stars but your asking for stars plus a summary and an statement. Already dropped off.
Interestingly, 5.6% of all orders end up leaving a review - which is VERY high. 3.2% would also be pretty good when the norm is maybe 1-2%. We haven’t seen a drop off in reviews since the new system went out.
If we were to show something like “Ships within 3 days,” as a calculated amount vs user input would be the reality. If you turn that into an input field (as we had at one point), the truth vs the input begin to conflict with each other. Showing the honest answer of what is calculated can’t be a bad thing from a customer standpoint.
So ultimately its a matter of balancing truth and expectations. No easy answers…