The Networked Bird Observatory Blog

As regular players are too-painfully aware, there are a couple of ways in which the CONE SF system has been “nerfed” (gamer slang for a system that has intentionally been made less-capable in some area).

One disability that the system has had from the beginning is the way the camera will automatically zoom to a fairly zoomed-out position (not the maximum zoomed-out position, but pretty far) if you try to point it in the area of the buildings in the upper lefthand corner of the panorama. As the FAQ explains, that has been done for privacy reasons, and while it’s kind of a pain when you’re trying to get a shot of the Rock Pigeons on the rooftops, I think it’s probably necessary.

The second disability is one that the game acquired on May 7. I talked about it some in System Changes. Basically, since that time, certain operations take a lot longer than they used to. In particular, loading the My Gallery page, and entering an ID on an individual photo page, can sometimes take a very long time.

My assumption has always been that this represents an intentional act of nerfing, rather than an unintended performance problem. (Update: Actually, as of a couple of days after I originally posted this, I no longer think this. I’m now thinking it’s an unintended performance problem. I’ll post more on that when I’ve got more specifics.) I’ve mentioned it a few times in emails to the site’s operators, and although I’ve received some feedback from Professor Ken Goldberg (the project’s co-director) and from Bryce Lee (one of the students who developed the system) to other comments I’ve made, they’ve never said anything specifically about this issue. Which is pretty much what I’d expect. There’s a complex dynamic that controls what information you do and don’t give out when you’re developing an online multiplayer game, and as someone who’s been on both sides of that issue I understand, and sympathize with, the CONE SF creators’ hesitation to share too many details about what they’re doing.

So, given that I don’t really know what the intention behind this was, or the specifics of how it has been implemented, it’s hard for me to be sure of what I’m about to say. But I’ll say it anyway: I think this particular act of nerfing isn’t very effective. It doesn’t appear to me that it’s accomplishing much beyond annoying a lot of us to the point that we’ll probably end up using the system less than we otherwise would.

In preparation for writing this item, I ran some tests to try to get a better idea of how the nerfing actually behaves. I turned off image loading in my browser (to better monitor the connections I was making to the server), loaded the My Gallery page multiple times in new browser tabs by Command-clicking in Firefox, and noted how long it took for the individual requests to complete.

Here’s the first test I ran, in which I submitted five requests in quick succession (spanning two to three seconds, probably, from first to last). For each request (numbered in order), I give the number of seconds until the server’s response completed:

Requests submitted at 08:30:00:

1: 47s
2: 54s
3: 1m 00s
4: 1m 08s
5: 1m 15s

I suspected that that test might have been influenced by the clicking around on the site I’d been doing just prior to that, so I waited several minutes, then tried again, this time only making two requests. The results:

Requests submitted at 08:35:00:

1. 8s
2. 16s

Then I ran a test with three near-simultaneous My Gallery page loads:

Requests submitted at 08:37:00:

1. 8s
2. 15s
3. 22s

Next I did a test with four:

Requests submitted at 08:39:00:

1. 8s
2. 16s
3. 21s
4. 28s

And then with five:

Requests submitted at 08:41:00:

1. 8s
2. 16s
3. 24s
4. 35s
5. 1m 35s

That one was interesting. Between the fourth and fifth request returning I suddenly got a huge jump of an additional minute of time. Two and a half minutes later I ran another test with five requests; here are the results from that:

Requests submitted at 08:45:00:

1. 49s
2. 1m 45s
3. 2m 45s
4. 3m 00s
5. 3m 05s

Again, there’s lots of extra time being injected into several of those responses. And then I tried 10 simultaneous requests:

Requests submitted at 08:49:00:

1. 8s
2. 16s
3. 24s
4. 30s
5. 35s
6. 43s
7. 50s
8. 56s
9. 1m 05s
10. 1m 10s

Now it seems to be back to “normal”, with about 7 or 8 additional seconds being tacked onto each request. And then, just now, after not using the system for an hour or so, I loaded the My Gallery page once, waited two minutes, and then loaded the page 10 times in quick succession. The response times were as follows:

Requests submitted at 10:11:00:

1. 8s
2. 15s
3. 22s
4. 29s
5. 34s
6. 40s
7. 47s
8. 54s
9. 1m 00s
10. 1m 08s

Again, all the responses look “normal”, with about 7 additional seconds (on average) for each additional request.

So, summarizing what I’ve seen, the nerfing system seems to work like this: requests for the My Gallery page are queued up, and responded to with one response every 7 seconds (or so). There’s also something that occasionally injects a large (and variable) amount of extra time, on the order of 45 seconds to a minute. I can think of a couple of explanations for that:

1. It could be that the nerfing is being done on a per-user-account (or per IP address, maybe) basis. In that scenario, the occasional injections of big blocks of additional time could be the result of some kind of longterm effect of my previous testing. Maybe the nerfing algorithm is integrating requests across a longer span of time than the minute or two of inactivity I was leaving between requests?

2. It could be that nerfing is being done on a server-wide basis. In this scenario, the injections of extra time could be the result of other users’ requests slipping into the queue ahead of mine.

I can think of a couple of ways to figure this out. One way would be to create a “sock puppet” account and run two tests simultaneously (from different IP addresses, I guess, to be sure I was isolating the tests from the server’s perspective). But I haven’t used sock puppets to try to figure out the system up until now, and don’t think it’s really kosher, even though I haven’t been able to find any language explicitly forbidding it on the system.

An easier way to test the “one big queue, across all users” theory would be to repeat my tests late at night, when I can be reasonably sure I’m the only person interacting with the system. Maybe I’ll try that.

In the meantime, I hope the good people on the CONE SF development team will get rid of the response-time nerfing. Especially when one is trying to snap a good picture of a rarity, taking lots of photos and then trying to delete the rejects in order to get another one before the bird disappears, the nerfing is very annoying. (That annoyance is multiplied many times over if you only have one or two shots left in your roll for the day.) If it’s true that all the users entering nerfable requests at any given time are competing with each other for a place in the queue, this gets even worse, since when a rarity is on-camera everyone will be trying to delete their rejected shots at the same time. In effect, the system will become cumbersome and unusable at the precise moment that players are most sensitive to its performance.

That’s not fun. And games are supposed to be fun. 🙂

Okay; done ranting for now.

kryptonkay got this interesting shot at 4:05 p.m. today:

It’s pretty clear, after some study, that it’s a female House Sparrow. But the posture and the camera angle are deceptive; it looks like a pretty unusual bird. This led to a number of wrong guesses being entered, such that by the time I came along and entered my own guess, it had been placed in status ‘Disputed’.

My ID of House Sparrow was enough to flip it to that ID, which is handy, because I’ve been wondering just what it takes to go from disputed to ID’d.

Before my vote the tally was like this:

Dark-eyed Junco: 1
Pygmy Nuthatch: 2
House Sparrow: 5

After my vote, it was:

Dark-eyed Junco: 1
Pygmy Nuthatch: 2
House Sparrow: 6

As most of us have probably noticed, two votes to one is sufficient to get a bird with only three votes classified as whatever got the two votes. And the evidence of this photo seems to indicate that the same principle works with larger numbers, too, leading me to believe that this is the general rule: A bird is given an ID if at least 2/3 of the votes cast agree on a particular species.

I took a look at some of the more notorious shots currently in ‘Disputed’ status, to see what it will take to get them classified “properly” (according to my personal definition). For instance, there’s this photo by bluebean, which I believe (for reasons I explained in more detail in Well, Hello Little Lady) to be a female Western Tanager:

Currently the vote tally for this one is:

Western Tanager: 10
Lesser Goldfinch: 4
Bullock’s Oriole: 3
American Goldfinch: 1

So, to get the image classified as Western Tanager would require six more votes for that, with no dissenting votes (because there currently are eight non-tanager votes, and we need to double that). I guess I won’t hold my breath for that; there’s probably a better chance of getting another, clearer shot of the bird if we want Western Tanager to be officially in the game. (Wouldn’t a shot of a male be awesome?)

Then there’s this photo by harpsichordgal, which I talked about in Is That a Hooded Oriole?

Currently its vote total is like this:

Hooded Oriole: 4
Black-headed Grosbeak: 4
Evening Grosbeak: 1

So to get that officially ID’d as the game’s first Hooded Oriole would require (again) six more votes for that, with no dissenting votes. Sigh. Probably not going to happen anytime soon.

Next up are these two shots, from the first (and so far only) appearance of the Lazuli Bunting, back on the morning of May 9. Both of these were by kryptonkay:

The vote tally on the first one is:

Lazuli Bunting: 5
Black-headed Grosbeak: 4

That means to switch that ID to Lazuli Bunting would require three more votes for that, with no dissenting votes.

The vote tally on the second image is currently:

Lazuli Bunting: 6
Black-headed Grosbeak: 3
Blue-headed Vireo: 1

That means to switch that ID to Lazuli Bunting would require two more votes for that, with no dissenting votes.

I really hope we can get one or both of those switched to Lazuli Bunting, for the following reasons:

1. That first shot was the first appearance of the bird in the system. It would be nice to have it properly classified.
2. The second shot is arguably the clearest one taken of the bird during his appearance.
3. Most importantly, of the other seven shots of the bird that were taken that day, and that were successfully ID’d as Lazuli Bunting, none were taken by kryptonkay. That’s not fair. She seriously deserves to have that bird in her list of photographed species.

If you haven’t entered a vote yet for one of these disputed photos, I encourage you to do so now. Well, as long as you’re willing to vote the right way. Otherwise, I encourage you not to bother. 🙂 Thanks!

Update: And now, thanks to some help from users lal and vireo, the second of those has been flipped to Lazuli Bunting. Yay! All is right with the world.

I hereby enter this shot, which I snapped at 05:38 today, into the competition for “earliest identifiable image”:

I haven’t actually checked yet, but I’d have to think that what with the days only slowly lengthening on their way to the summer solstice, and with Robins being notoriously early risers, that this photo is going to be tough to beat, at least for a while.

I thought I’d set myself a challenge today, to see if I could get ten photos featuring ten different species. Sort of a “mini big day.”

Here are the birds I got, and the times at which I got them:

1. 06:03 – House Finch
2. 06:46 – Chestnut-backed Chickadee
3. 08:04 – Rock Pigeon
4. 08:06 – Dark-eyed Junco
5. 08:20 – Pygmy Nuthatch
6. 08:34 – House Sparrow
7. 10:27 – American Robin
8. 12:00 – Anna’s Hummingbird
9. 13:21 – Mourning Dove
10. 16:55 – Western Scrub-Jay

I actually was on track to complete the ten birds much earlier; a Scrub Jay that would have been my 10th species visited the ball feeder at 13:48, and over the next three minutes other players snapped 10 different pictures of it. But I got an important phonecall during that time, and my back was to the computer, and I didn’t notice it. When the call ended and I saw what I’d missed, I mentally kicked myself, then decided that I needed to get a life, and went out to watch some real birds. I didn’t get back to the computer until late in the afternoon, so I also missed all the fun today when someone found a Mourning Dove perched on what I think of as the “Upper Limb”, that big tree branch in the middle of the forest that is up above that other big branch (which I think of as just “The Limb”) where I’m always hoping to see something big and impressive perched. A Red-shouldered or a Cooper’s Hawk, maybe; either one of those would do nicely.

Anyway, there you go: ten species in ten photos, completed as of 16:55. Consider the gauntlet thrown down, fellow players. Let’s see you beat it.

So, I’ve done some digging around in the list of players and their scores, trying to reverse-engineer how the scoring system works. I think I’m pretty close to understanding it; a few parts of this are guesses, but they seem like reasonable guesses based on what I can see.

When you first take an image, that image is only visible to you (in the “My Gallery” view). This is the interval during which you can delete the image if you don’t want it to count against your 10 shots for the day. If you go ahead and add an ID to the image, or add a comment to the image, you lose the ability to delete it, and it immediately becomes publicly visible (Correction: Nope. It stays visible only to you until 10 minutes after it was taken, at which point it becomes publicly visible. You can see it in the “Public Gallery” view during that 10 minutes, but no one else can.)

Once the photo is publicly visible, someone else is likely to add a comment or ID to it very quickly, at which point the image will no longer be delete-able.

You score points as follows:

1 point: Taking a photo (any photo).

3 points: Being the first to supply an ID to a photo. Note: I’ve seen cases where it looks like the user gets 3 points for voting first, even if the ID is subsequently overruled by other voters.

2 points: Being the second or third user to supply an ID to a photo.

That appears to be it, as far as scoring goes. I haven’t found any evidence that there’s any extra weight given to “rarities” (birds with few appearances in the system). Nor have I found any evidence that being correct with your IDs makes any difference to your score. Being the first to correctly ID a photo where the first person to vote voted incorrectly does make it so you get credit for the ID in your “Classifications” section, but I don’t believe it gives you 3 points; in that case I think you still only get 2 points. I haven’t actually tested that, though.

It does seem to be the case, though, that your points for being one of the first three voters only get credited to you if the photo eventually manages to get three votes. So if, say, you took a bunch of photos of blackness in the middle of the night and voted all of them as owls of various unlikely varieties, you would only get your 3-points-per if at least two other users came along and voted ID values for those images. If other users did the sensible thing of simply entering “no bird” descriptions, you wouldn’t get any points.

Speaking strictly hypothetically, of course. 🙂

Note that this is contradicted by the tutorial on the site, which says:

If other users have classified the image, you will then see what their guesses were. If a majority of users agree, then the image will move from the Unclassified pool to the Classified pool, and you will receive additional points for making a correct classification. (emphasis added)

So, if you take 10 photos per day of identifiable birds, and immediately apply an ID to them, that’s 40 points per day you are guaranteed to get (10 for the 10 photos, plus 30 more for the 10 first IDs). The people currently atop the leaderboard (wyoming is the current leader, with 3,303 points, for a daily average of 367 points during the 9 days he or she has been playing) appear to be racking up those scores mainly by being extremely zealous at patrolling the “Public Gallery” view’s “Not Yet Classified” page, getting their IDs in very quickly whenever a new image by another user becomes available.

The most surprising thing to me about this scoring system is that it apparently doesn’t matter at all if you’re particularly good or bad at identifying the birds. You just have to be fast, and diligent. But obviously there are less-quantifiable rewards for getting the IDs right (like building and maintaining a reputation for being knowledgeable with the game’s other participants). And there are other ways to have fun with the system besides just trying to grind out the highest score; as with bird-watching itself, there are aesthetic and educational rewards to be had that outweigh the pleasures of winning some crass competition.

At least, that’s what I tell myself as I look at those players with the astronomically higher-than-mine scores.