User Tag List

Thanks useful information Thanks useful information:  77
Page 2 of 4 FirstFirst 1234 LastLast
Results 21 to 40 of 80

Thread: New Vote/Rating System Feedback thread

  1. #21
    Member
    Join Date
    22 Jun 2010
    Location
    Lake Macquarie
    Posts
    4,883
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by CAP View Post
    To my mind it's still a competition so am I correct to compare and judge all other pics against what I perceive to be the best shot put forward for that week?
    Exactly! In fact the new guidelines make it clear that entries should be judged competitively for that week rather than against an arbitrary criterion for what is a good or bad image in the mind of the voter. It is a comparison scoring system, not an all-time best rating system.
    Last edited by WhoDo; 26-09-2011 at 1:57pm.
    Waz
    Be who you are and say what you mean, because those who mind don't matter and those who matter don't mind - Dr. Seuss...
    D700 | D7000 | Nikkor AF-S 18-55 DX 1:3.5-5.6G | Nikkor AF-S 55-300 DX 1:4.5-5.6 G ED | Nikkor AF 50 f/1.8D | Optex OPM2930 tripod/monopod | Enthusiasm ...
    My Flickr images ...

  2. #22
    Still in the Circle of Confusion Cage's Avatar
    Join Date
    25 May 2010
    Location
    Hunter Valley
    Posts
    5,454
    Mentioned
    3 Post(s)
    Tagged
    0 Thread(s)
    I concur with Allann's comments above.

    I didn't necessarily give my top pick a 10 if I didn't think the shot was worth a 10.

    I feel each photo should be judged from 1 -10 on it's individual merit, and not picking the best of the bunch as a '10' and down-rating all other entries.

    This method should stop the scenario where one not particularly outstanding photo is the best of a rather ordinary bunch of entries for that competition, and receives the maximum 10 score from all voters. When looked at in the context of one monthly winner scoring maximum votes and being overshadowed by eleven other monthly winners with better quality entries and lower scores, it seems rather inequitable.
    Cheers
    Kev

    Nikon D810: D600 (Astro Modded): D7200 and 'stuff', lots of 'stuff'

  3. #23
    Member CAP's Avatar
    Join Date
    02 Nov 2006
    Location
    Wollongong
    Posts
    1,835
    Mentioned
    1 Post(s)
    Tagged
    0 Thread(s)
    I quite like the idea of automatically sorting the images on screen as they are scored, it would definitely help the voting process.
    While we at at it and Kym is busy punching code like crazy.... would be be worthwhile to impliment this scoring system into the everyday forums like Constructive Critique?
    A "Rate My Photo" option that will tally the averaged scores as they happen.
    This option could be either permanently enabled or selectable option to enable by the thread starter if there are some that choose not to.
    My thinking that if someone puts forward a score out of 10 for a pic in CC then they might be more inclined to say more than "nice shot" (of which I am guilty on numerous occasions)
    CC always welcome and appreciated.
    Tweaks welcome but please add how and why.



  4. #24
    Member
    Join Date
    23 Jan 2009
    Location
    Hobart
    Posts
    569
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    The problem with giving the best image an 8 out of 10 (rather than 10 out of 10), is that your definition of 10/10 may be vastly different to someone else. By giving the best image in the competition a score of 10 it helps keep the scoring relative, and a little bit more consistent between voters.

    Andrew.
    https://forkandfoot.com

    Nikon D7000
    Sigma 10-20mm ~ Nikon 18-55mm ~ Nikon 55-200mm ~ Manfrotto 190XProB ~ Manfrotto 488RC2
    ~ LowePro Top Load Zoom 1


  5. #25
    Amor fati!
    Join Date
    28 Jun 2007
    Location
    St Helens Park
    Posts
    7,275
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    i like it. you vote for every entry giving it a score out of 10. top stuff.
    I found with the old system id scroll thru and if i liked one i'd tick it and after 5 ticks if there was still one down further i liked id have to scroll back and forth comparing the two and maybe untick one before i could tick another....

    in shot, much easier!

  6. #26
    Member alextdel's Avatar
    Join Date
    11 Nov 2009
    Location
    Gold Coast Queensland
    Posts
    96
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Thanks for the guide, I found it very useful.

    I took longer than normal to consider the photos, this will probably improve with practice but the voting could get cumbersome if there were many entries. I felt that the pictures were relatively 'right' for my judgement - again the guide was handy here.

    Forgive me if this the following has been discussed previously. Coming from a tech background (! ), I know that using measures of average (median) are sensitive to relatively few extreme values. That is a few, even one, extreme value has much more effect than many non-extreme values. This effect is potentially more significant the less people are doing the scoring. A major concern here is the case of voting for your own picture. I assume most people would vote high for their own photography despite wanting to be fair-minded for the overall process - after all, we put a photo in the comp because we think it is worthy.

    For example, assume 20 Ausphotographers have given their score for a photo and the average of the 20 scores is 5 (total of the scores would be 100). Now another 4 people log in and give the same photo scores of 4, 6, 5 and 5. The result would be that the average score (rank) remains at 5 [(100+4+6+5+5) divided by 24]. There doesn't seem anything wrong with this as the photo looks like it is probably a picture worthy of a 5 score. But now a single new user logs in and gives the photo a 10. The mean now rises to 5.2. So one voter has had more effect (4%) than the previous four voters (0%).

    The sensitivity of the median to extreme values is also the case for low votes. Potentially relatively few scores of 1 could jeopardise an otherwise reasonable score average.

    I think that 'voting' for yourself is ok but the scoring for yourself could be an issue. I do not suggest for one second that any of our fellow Ausphotographers are deliberately unethical (quite the opposite is why I like the site), but in fact the fairest and most ethical competition user will probably be disadvantaged if they accurately judge their own photo compared to others who automatically give their own photo a 10. Now in the case of a fabulous and obvious winner I think the best photo will still get the best rank, but in close run contests voting for your self and strategically voting low for other close competitors could tip the balance.

    How much this all matters, I don't know. I also know the site admins do a good job of moderating the competitions for abnormal practices. I still applaude the effort to increase the value of the competitions to all entrants.
    Alex Delaforce - Teacher / Education and Technology Manager, Gold Coast
    Canon 6D, Canon 50D (died), Tokina 12-28, Tamron 70-200 (VC), Sigma 50.

  7. #27
    Member
    Join Date
    22 Jun 2010
    Location
    Lake Macquarie
    Posts
    4,883
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by trublubiker View Post
    I feel each photo should be judged from 1 -10 on it's individual merit, and not picking the best of the bunch as a '10' and down-rating all other entries.
    Individual merit against what criteria, Kevin? It's a competition for THAT WEEK ONLY, so you are rating say 40 entries on a scale of worst (1) to best (10) in that company only. Sure there is nothing stopping you from saying "I don't think any of these deserves to win", in which case you won't allocate a 10, but you need to remember that everyone else will rate their winner for that week a 10, and if it isn't the same image you think is best then giving yours less than 10 is robbing it of some comparative value.

    Quote Originally Posted by trublubiker View Post
    This method should stop the scenario where one not particularly outstanding photo is the best of a rather ordinary bunch of entries for that competition, and receives the maximum 10 score from all voters. When looked at in the context of one monthly winner scoring maximum votes and being overshadowed by eleven other monthly winners with better quality entries and lower scores, it seems rather inequitable.
    Well first we are talking about POTW Qualifying only at the moment. The final choice is still a pick from 4-6 entries. Secondly, there will always be some POTW or POTM entries that outshine their peers; that's why there is a POTY from all of the winners! POTW is about picking the best photo submitted THIS WEEK, not this month, this year, or compared to whatever else I may have seen previously in comps here, etc. I hope that explains the intent a little better, mate. No disrespect to your opinions intended.

  8. #28
    Member
    Join Date
    22 Jun 2010
    Location
    Lake Macquarie
    Posts
    4,883
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by alextdel View Post
    I know that using measures of average (median) are sensitive to relatively few extreme values.
    Yes, Alex, we discussed the statistical impact of outliers and decided the effects were too small here to justify a system of discarding high and low votes as they do with diving, skating, etc. The ratings only apply to qualifying so the chances of affecting the outcome are are ~1% and too small to justify the programming effort at the moment. I'm sure Kym and Rick will be monitoring the system for fairness, but it already beats one where you are forced to discard images of equal value that sit at the margins; i.e. vote or not vote.

  9. #29
    It's all about the Light!
    Tech Admin
    Kym's Avatar
    Join Date
    15 Jun 2008
    Location
    Modbury, Adelaide
    Posts
    9,643
    Mentioned
    23 Post(s)
    Tagged
    0 Thread(s)
    Thanks Waz. Yes, that is why we are keeping the final single selection vote.

    Even if 50 voters (and we usually have 80+) give 7 pts and 10 more give 8 and 10 give 6 (mean of 7) and the entrant gives themselves 10 points....
    The total is 7.042 - meh! No drama.

    In looking at the actual voting trends I see a good spread and consistency; so it appears to be very fair.
    Even at the extremes they seem to balance out in a normal curve sense, i.e. for each low vote there is a high vote.

    FYI Rick and I have a tool that shows all the raw voting down to each individual vote in a table,
    and we do monitor voting for irregularities, esp. when prizes are involved.

    Remember not everyone sees an image the same way - there will be variations in scoring
    I don't agree with the camera club judging about 1/3 of the time, but don't complain either as the judge has their own views.

  10. #30
    Member
    Join Date
    26 Nov 2008
    Location
    Booval, Qld (near Ipswich)
    Posts
    2,018
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by CAP View Post
    would be be worthwhile to impliment this scoring system into the everyday forums like Constructive Critique?
    A "Rate My Photo" option that will tally the averaged scores as they happen.
    This option could be either permanently enabled or selectable option to enable by the thread starter if there are some that choose not to.
    I love this idea too... especially with a opt in/out when creating the thread. That would save the OP asking for CC all the time. And it'll make the 10-1 scoring system more natural for people to use, and we would start to get an understanding of what the scores actually mean.

  11. #31
    Member
    Join Date
    12 Feb 2008
    Location
    Brisbane
    Posts
    7,831
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by Allann View Post
    I love this idea too... especially with a opt in/out when creating the thread. That would save the OP asking for CC all the time. And it'll make the 10-1 scoring system more natural for people to use, and we would start to get an understanding of what the scores actually mean.
    I've suggested this before too, perhaps flippantly. Even a gold/silver/bronze breakdown for fun. Add up the number of points per month and get a monthly merit etc - this would also encourage perhaps more pictures being posted
    Darren
    Gear : Nikon Goodness
    Website : http://www.peakactionimages.com
    Please support Precious Hearts
    Constructive Critique of my images always appreciated

  12. #32
    It's all about the Light!
    Tech Admin
    Kym's Avatar
    Join Date
    15 Jun 2008
    Location
    Modbury, Adelaide
    Posts
    9,643
    Mentioned
    23 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by Allann View Post
    Quote Originally Posted by CAP View Post
    would be be worthwhile to implement this scoring system into the everyday forums like Constructive Critique?
    A "Rate My Photo" option that will tally the averaged scores as they happen.
    This option could be either permanently enabled or selectable option to enable by the thread starter if there are some that choose not to.
    I love this idea too... especially with a opt in/out when creating the thread. That would save the OP asking for CC all the time. And it'll make the 10-1 scoring system more natural for people to use, and we would start to get an understanding of what the scores actually mean.
    Interesting idea!

    If I ever got around to it ... It would work with single image threads.
    It's too difficult/messy to do this per image due to attachment vs linked image.
    It would amount to an optional anonymous thread rating system.

    It's not going to happen for some time, if we decide to even implement it; bedding down the new voting is far more important.

  13. #33
    Member
    Join Date
    07 Dec 2008
    Location
    North Haven
    Posts
    607
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Thumbs up from me guys.

    I found it simple and quick because I wasn't scrolling up and down.

    I did have trouble with the page timing out when I clicked on the vote button, in both chrome and firefox last night but tonight in firefox it worked fine.
    Cheers
    Wilky "We have taken a vow of poverty, send money to annoy us".

    I know you believe you understand what you think I said,
    But I am not sure you realise that what you heard , is not what I meant

    Canon Eos 450D Linux user

  14. #34
    Member rene52's Avatar
    Join Date
    18 Nov 2009
    Location
    Benalla
    Posts
    1,008
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Well I am sorry to be one of the few that don't think this is better. I agree with the problem of only havig 6 votes to decide on previously being difficult but this is time consuming at its peak. I have been critiquing as much as possible which again takes time but I assume that a critique is better than a vote where there is no feedback at all. I for one will now look at only critique not weekly photos (mind you I haven't had time to put photos in anyway so I suppose nothing lost or gained). Will wait to see the next iteration of voting.

  15. #35
    Administrator
    Threadstarter
    ricktas's Avatar
    Join Date
    24 Jun 2007
    Location
    Hobart
    Posts
    16,316
    Mentioned
    11 Post(s)
    Tagged
    1 Thread(s)
    Quote Originally Posted by rene52 View Post
    Well I am sorry to be one of the few that don't think this is better. I agree with the problem of only havig 6 votes to decide on previously being difficult but this is time consuming at its peak. I have been critiquing as much as possible which again takes time but I assume that a critique is better than a vote where there is no feedback at all. I for one will now look at only critique not weekly photos (mind you I haven't had time to put photos in anyway so I suppose nothing lost or gained). Will wait to see the next iteration of voting.
    I agree that it is a bit more time consuming than picking 4/5/6. Wait till Sunday evening when we reveal the polls and members can see their rating. I think that is where the member benefit will be. Members will then be able to see how their fellow AP members rated their photo. I think the benefit of a 6.182/10 will offer more in the way of valuable feedback, than the current, you got 6 votes, system.

    Certainly forum critiquing is also an integral part of AP, and we are trying to improve the value of a vote, in the competitions, to where the rating reflects how good an entry was (for all entrants), rather than a simple, number of votes. By comparing their rating to the guidelines a member can see how other rated their photo and use the guideline description to get a better understanding of how to improve it. The current you got 2, 6, 12, 20 votes doesn't give entrants that feedback in any way.

    Next weeks POTW will use the 'old' voting system, and then the week after we will again trial the new vote ranking system. After that members will have had a good chance to evaluate both systems and see the results of how a ranked vote gives more feedback to the entrants, and then we will put it to members as to which system they want, or both (we can still have both systems and apply them to different competitions).
    Last edited by ricktas; 26-09-2011 at 6:58pm.
    "It is one thing to make a picture of what a person looks like, it is another thing to make a portrait of who they are" - Paul Caponigro

    Constructive Critique of my photographs is always appreciated
    Nikon, etc!

    RICK
    My Photography

  16. #36
    It's all about the Light!
    Tech Admin
    Kym's Avatar
    Join Date
    15 Jun 2008
    Location
    Modbury, Adelaide
    Posts
    9,643
    Mentioned
    23 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by Kym View Post
    Quote Originally Posted by Allann View Post
    I liked the new voting system. But I think this could improve things;
    • At the confirmation stage, can the images be sorted by the score given (10 to 1) so you get a preview of your rankings before confirming the vote, otherwise the confirmation is quite tedious.
    Great idea (one software guru to another ) ... I'll implement that later in the week.
    Done!

  17. #37
    Shore Crawler Dylan & Marianne's Avatar
    Join Date
    21 Mar 2009
    Location
    Adelaide
    Posts
    8,654
    Mentioned
    1 Post(s)
    Tagged
    0 Thread(s)
    I found it pretty easy and as someone who usually thumbs images I thought were non contenders, the number of clicks was no more or less
    I must admit though, I made the mistake of judging the images on standalone merit and not with regard to each separate competition so I didn't use the entire spectrum of marks (next time I'll read the voting guidelines closer!)
    I also like the fact that you get the intermediate feedback - after all, an image that everyone thinks universally scores 7-8 might well have got no votes because it just looked good to everyone but not unique.
    Last edited by Dylan & Marianne; 26-09-2011 at 7:06pm.
    Call me Dylan! www.everlookphotography.com | www.everlookphotography.wordpress.com | www.flickr.com/photos/dmtoh
    Canon EOS 5dmk3 : 17-40 F4 L, 70-200F2.8 canon L, 24-70mm canon L, Gitzo Safari +1178 ballhead. |Canon 5dmkII, 16-35mmF2.8 II L, Gitzo 2541 )
    Singh Ray/Hitech/Lee assorted filters, Z pro modified system Cokin holder
    Post : Lightroom 3.6 catalogue -> Export as 16bit TIFF, Edited CS5 -> resized for web.

  18. #38
    It's all about the Light!
    Tech Admin
    Kym's Avatar
    Join Date
    15 Jun 2008
    Location
    Modbury, Adelaide
    Posts
    9,643
    Mentioned
    23 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by dtoh View Post
    I must admit though, I made the mistake of judging the images on standalone merit and not with regard to each separate competition so I didn't use the entire spectrum of marks (next time I'll read the voting guidelines closer!)
    You should ideally have one 10 and probably none <5 points, assuming all entries meet the theme and are technically ok.

  19. #39
    Member rene52's Avatar
    Join Date
    18 Nov 2009
    Location
    Benalla
    Posts
    1,008
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by ricktas View Post
    I agree that it is a bit more time consuming than picking 4/5/6. Wait till Sunday evening when we reveal the polls and members can see their rating. I think that is where the member benefit will be. Members will then be able to see how their fellow AP members rated their photo. I think the benefit of a 6.182/10 will offer more in the way of valuable feedback, than the current, you got 6 votes, system.

    Certainly forum critiquing is also an integral part of AP, and we are trying to improve the value of a vote, in the competitions, to where the rating reflects how good an entry was (for all entrants), rather than a simple, number of votes. By comparing their rating to the guidelines a member can see how other rated their photo and use the guideline description to get a better understanding of how to improve it. The current you got 2, 6, 12, 20 votes doesn't give entrants that feedback in any way.

    Next weeks POTW will use the 'old' voting system, and then the week after we will again trial the new vote ranking system. After that members will have had a good chance to evaluate both systems and see the results of how a ranked vote gives more feedback to the entrants, and then we will put it to members as to which system they want, or both (we can still have both systems and apply them to different competitions).
    Thanks for your comments on my feedback - I just feel that even the rating (well in my opinion ) doesn't really help me to take a better photo. To do so in most cases you need to be in the right place at the right time and also have the right equipment. I am by no means a professional photographer and as such would prefer to have a critique rather than just a rating. I do realise that without comps there isn't much to get excited about but for me this makes me spend more time where I could have spent the time giving feedback to people that post photos.

    I find that most of the photos that are presented at the POW are of a high standard anyway and as such would not find much to 'fix'. Now I am not writing this to complain or anything it is just it seems to difficult and time consuming. I am a member of quite a few sites that have photos posted and ask for people to discuss and spending extra time will not be something that would like to do.

  20. #40
    Administrator
    Threadstarter
    ricktas's Avatar
    Join Date
    24 Jun 2007
    Location
    Hobart
    Posts
    16,316
    Mentioned
    11 Post(s)
    Tagged
    1 Thread(s)
    Quote Originally Posted by rene52 View Post
    Thanks for your comments on my feedback - I just feel that even the rating (well in my opinion ) doesn't really help me to take a better photo. To do so in most cases you need to be in the right place at the right time and also have the right equipment. I am by no means a professional photographer and as such would prefer to have a critique rather than just a rating. I do realise that without comps there isn't much to get excited about but for me this makes me spend more time where I could have spent the time giving feedback to people that post photos.

    I find that most of the photos that are presented at the POW are of a high standard anyway and as such would not find much to 'fix'. Now I am not writing this to complain or anything it is just it seems to difficult and time consuming. I am a member of quite a few sites that have photos posted and ask for people to discuss and spending extra time will not be something that would like to do.
    Sure, there are members on AP who enter every comp, but only post to the forums about once every couple of weeks. We know we can't have all of the site, be everything, for each member. In the end we are trialling an idea to improve the current vote system. Each member is free to either join in on the comps, forums etc, or not, as they wish.

Page 2 of 4 FirstFirst 1234 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •