Introduction:
The idea behind this draft blog is to introduced a system to give Credibility Score (CS) to each subscriber to any social network or collaboration portal based on their contribution to the community. Their contribution here might be posting an article or a review, how the system builds CS for that user will be based upon subsequent reactions by other members and the amount of posts performed by that user.
How it works:
Whenever a user make any post of any kind in a social network, other users using this network will be presented with a rating mechanism to rate this post, the values used to rate the post are listed below and given weight for each category. By using all ratings the user has got for his post, the below process takes place to calculate CS for this post.
Credibility Score of a user is the sum of all CS for each post they earn. CS of a user can be then used to give weight for subsequent posts of the user, CS is a number between 0 and 1, where 0 means the user has no credibility and 1 means they have the maximum credibility.
Ratings:
Each rating will be allowed one of the following values:
| Rating | Weight |
|---|---|
| True | 1.00 |
| Mostly true | 0.75 |
| Half true | 0.5 |
| Mostly false | 0.25 |
| False | 0 |
| Ridiculous | -1 |
Review credibility score:
- For each review calculate total rating and divide them into 6 categories: true, mostly true, half true, mostly false, false and ridiculous
- Calculate proportion value and credibility score for each category, for example, if a review has 29 ratings:
Category Category weight Category count Proportion Credibility True 1 4 0.13 0.13 Mostly true 0.75 16 0.55 0.41 Half true 0.50 7 0.24 0.12 Mostly false 0.25 2 0.06 0.01 False 0 1 0.03 0 Ridiculous -1 0 0 0 0.67 - So the user has total of 0.67 credibility for this review
User credibility score:
- Get total number of reviews committed by a user (for example a user has 5 reviews)
- Get the total number of rating for user reviews (for example a user got 3 out of 5 reviews rated)
- Calculate evaluation ratio (number of reviews rated / total number of reviews), for our example it’s ⅗ = 0.60
- Calculate average credibility score for all rated reviews, for example of those 3 rated reviews got credibility scores: 0.67, 0.25 and 0.41 which is
- (0.67 + 0.25 + 0.41) / 3 = 0.44
- User unnormalized credibility score is 0.44
- Multiply evaluation ratio with unnormalized credibility score to get the final credibility score for a user, so
- User credibility score: 0.60 * 0.44 = 0.26
- The more ratings a user get the higher their credibility score preserved
Note: might have issues for example if a user have only 1 review rated while they posted 75 reviews, so it doesn’t really reflect their true credibility
So for evaluating ratings done for schools, credibility score for a user will affect this rating by multiplying each rating done by the user with his/her credibility score, for example if a user give the following ratings:
| Category | Rating (out of 5) | Rating after applying CS |
|---|---|---|
| Cleanliness | 4 | 1.04 |
| Transportation | 5 | 1.3 |
| Teachers | 3 | 0.78 |
References: