New Computer Vision Model (v2.11) with 1,256 new taxa

91 plant species from southern Africa (although 10 of these are exotics). Nice to get more of our flora onto the AI.

@lj_lamera Hecuba scortum is now in the CV. That's mostly due to your efforts! (I followed your lead, and @gavin_n did a lot as well, much of it before we turned our sights to the species).

@lj_lamera there are 99 research-grade observations of A. chemnitzii. I think the next update will have it.

Question. First, on an obs. that looks like grass without insects that the observer made a non-CV ID for a bee family for, when I then clicked on CV the suggestions were wasps, flies, etc., not plants. I then added a non-CV Grasses family ID. Now, when clicking CV, the suggestions are for grasses/plant species. So, does this mean the CV is partly basing it's guesses on the IDs users make, and if so does that apply for both CV-based IDs or only non-CV-based IDs? In the event either is true, I was never aware of this. It would also be unclear if either would be ideal, since an initial incorrect user ID would seem to be misleading the CV, leading to a cascade of incorrect ID suggestions. Or, is there just something atypical going on with this particular obs.? https://www.inaturalist.org/observations/32769190

Also, as I typically say when I occasionally comment on CV update announcements, it's good that more taxa are being added and that the training is said to be improving, although from my day to day ID of wasps and bees (although they are cryptic groups) I typically notice no clear improvement, and occasional "way off" suggestions. Even after I correct thousands of incorrect CV and non-CV wasp or bee over months (e.g., a large percent of Europe Eumeninae obs. previously had been identified to the wrong genera), this remains seemingly unchanged. This would seem to suggest that while it's good more taxa are being added and that the CV is continuing to train, it will probably be many years before it's ID accuracy noticeably improves, at least for somewhat cryptic groups, or taxa with very variable phenotypes including coloration, like fungi.

This would also be why if you upload a clear image of a bird, then add an ID of a flower, CV will not know what to do and turn up with no suggestions. The only reason CV is able to recommend insects for the grass photo is because a many insect photos are taken in the grass. The same thing happens when you add a bird ID to a tree photo.

The Suggestions system currently automatically filters by the observations' ~12 iconic taxon (e.g. Birds, Mammals, Insects, Plants, Mollusks, etc.). As you say, this was meant to help with situations where the CV doesn't know what the subject is (e.g. a bee on a flower). If there are IDs of bee, the iconic taxon will be Insects and the CV will filter suggestions accordingly

yes - we train the Geo Model and the CV Model on the same set of taxa.

First, was this ever, at least often, explained to people that CV changes it's suggestions based on IDs that are added? Or is this a new or newer change to how CV used to work?

When CV does do that, does it make a difference between whether the user's ID was a CV-ID or non CV-ID?

I'm actually still unsure if that's the best system. Consider a hypothetical alternative "more basic" system. In my linked real example, CV guessed grass was insects (as said, possibly because insect photos often have grass and the photo was a little unclear or confusing. But whether the next person adds a Grasses ID or an Insects ID, it's unclear why it's necessarily ideal for CV to become influenced/misled by that. Because if the user IDs an insect, that further misleads the CV, possibly making it then (when clicked on again) make more precise or confident insect IDs, even though it's still just grass. In an alternative system, the CV would just guess insects from the beginning (as it did), and that suggestion would remain the same even after users added additional grass and/or insect IDs. So, in that case we'd just say the CV suggested an incorrect ID (regardless of reason). Has such an alternative system ever been considered or tried?

Three other notes. It would be ideal for the guidelines to no longer say not to call the CV AI (artificial intelligence), while they could still indicate that the staff originally called it ComputerVision / CV. No offense to whoever coinced ComputerVision/CV, but it's a term with a very unclear meaning, especially when used as the abbreviation CV (e.g., epsecially for users in some other continents that tend not to read the Help docs or read the forum. Overall at least, many users have no ID what CV or the more cumbersome full term ComputerVision means, which means they don't understand comments that refer to it or have to ask people what it means.

Versus, AI/A.I. is a well known abbreviation, and remains accurate. It's unclear and unproven why calling it AI (or something like "auto-suggestions") is wong. Plus, it's learning in large part through our human IDs, so is an AI in some senses but also one that we (not only developers but also identifiers) have helped design and are helping learn. That should give users at least some partial "stakeholder" or "ownership" over whether to call it AI. It's also frustrating that for those of us who do currently call it AI, that people could think or say that we're doing something wrong per the guidelines.The other question as mentioned, is what to do about the fact that CV often has very inaccurate suggestions for cryptic groups (e.g. bees and wasps) and groups that have a lot intraspecific and interspecific variation in structural patten or coloration. For e.g., oftne generating a list of species suggestions all from different genera, while at the same time highlighting only one of them as "we're pretty sure it's this." As I've mentioned, I corrected maybe 80-90% of the 50% or so inaccurately user- or CV-misidentified European genera, but the CV accuracy has hardly if at all improved. A natural solution would be to first detect which taxa are cryptic, indicated by the CV making more mistakes or having more uncertainty about them, and then using broader or less certain CV suggestions for those taxa.

For e.g., if an observation is a somewhat unclear bee or wasp, the CV wouldn't say "we're pretty sure it's this taxon," and the other suggestions they give (at least the first suggestions) would be broader, e.g., listing Aculeata, Apoidea, Bees, Vespidae, or Crabronidae. Or at least mostly list genera (at least the irst suggestions vs. species. Finally, in such cases, the message at the top could be "we don't have a clear guess of what this is, so please take caution when consideing the possible CV suggestions. Lastly, it can be noted and taken into account that very blurry or distant photos are inherently harder to ID, so CV could even say soneting like "are you sure an organism is present?" or that "we have little to no confidence in making suggestions (or make none)," and finally that in those cases the obs. would be excluded from the CV's training. As a side note: the guidelines and admin doing more to recommend observers take clearer photos and crop them wheere necessary would help the CV's training. A large part of why the CV currently makes so many inaccurate ID guesses must be due to it's training including the difficult photos. And I'm aware that the CV matter is complex, but still think these suggestions (or something like them) would be ideal to consider implementing in some way.

Okay, that's good to know, although my opinions about it still remain the same. Note also that I said most people (users) don't initially know what it means, even if it's considered well-defined. Most immediate internet searches I find also called it computervision, Computervision, or Computer Vision, not ComputerVision. The last term's formatting sounds more confusing or unfamiliar, in my opinion. I'd also still argue that aside from people who are already familiar with the term computervision outside of inat, that most would assume it was coined here, and that in general the term itself (even in outside sources) is inherently vague, i.e., reading the term doesn't tell you what it is. Even misleading. Regardless (a separate question), isn't it fair to remove from the guidelines that we should call it CV and not AI? Not everyone associates AI as a bad thing, anyway.

I'd also be interested if anyone replies to my other questions or points above, which involve CV but don't involve whether to call it CV or AI. Is it possible that I have any valid points about the grass observation example? Has the alternative systems I suggested ever been tried/tested, or could they be? etc.

This also applies to loarie's recent estimation of RG accuracy being 95%, which is a clear overestimate (see the user comments under those inat journal/blog posts to see why). If we instead incentivized better (clearer) photo standards, we'd be helping the AI improve, but without doing so we may not be significantly increasing it's accuracy. Of coure, RG % accuracy and AI % accuracy (which, confusingly, we must now separate into multiple categories: AI guesses before user IDs, AI guesses after user AI-guesses, and AI guesses after non-AI user guesses) aren't identical, but do overlap and interrelate. Overall, there should be some positive correlation between RG % accuracy and CV % accuracy, since the former improves the latter.

I'm a bit confused as to why you expect me to answer the other questions in your earlier comment on topics such as how/when the AI started changing its IDs based on non-CV user IDs

But the comment you did make had an authoritative tone, leading me to infer that you (who are an admin, as a mod and curator) at least might know more about CV and could answer more questions about it. You also seemed to refer to my question (can we call it CV or AI) without noting whether you know or have a preference, which is just confusing.

My comment also has more relevance in the context of our past discussions. E.g., You seeming to mostly only make critical comments to me, while omitting replying to other parts of my comments or saying anything positive about me. Of course, if you're busy one day, or going to sleep, I don't blame that. Still, if/when you get time, you (or others, as I did say) could hypothetically at least reply (to the best of your knowledge) to my other questions. It can be recognized that the way you commented could (but might not for all people) lead people to think that it's a conversation-stopper. Overall, I'm not really blaming you on this specific page.

As for for the (in my memory) former guideline of "please call it ComputerVision and not AI" (something like that), I'd assume that you would know and remember that even more than me, despite not working on CV (or can I call it AI now?). Regardless, if you'll discuss (the things you might or do know) or wouldn't prefer to discuss them here, either way is fine with me, no problem. By the way, some of the overwhelmingly negative comments made by certain mods, staff, etc. (with little to no positive comments) may have a negative impact on whether I can even get accepted into a PhD program (iNat is like part of our CV for some of us). "Cheers mate."

In my memory, which I think is good on this because I strongly disagreed with it along with others like JSA (although I don't know if he knew it was once a guideline), the guidelines once said something like please don't call it AI.

For people working on the AI, my questions would be have each of the alternatives I've suggested been tested? If not, can they be tested? It's not necessarily clear that that the current system of the AI following users' non-CV-based IDs (and maybe also CV-based IDs) to make more precise related guessed IDs based upon them is an improvement vs. a worsening of accuracy, for reasons I gave in the earlier comments. Also, misleadingly, that would suggest that the AI is "following" the identifiers' ID guesses even within each observation, but identifier guesses can be correct or incorrect. Maybe the real reason this is being used is because the initial AI guesses, before users make IDs, are so inaccurate to begin with. And I do think the AI is currently very inaccurate.

But on a slightly different but related note, that makes the whole common sense/often portrayed functioning of the AI misleading. The AI is typically portrayed as a visual recognition identification guess. For example, on iPhone, if someone takes a photo of a plant there's now a (iPhone, non-inat app) option where an AI will guess what it is (with varying accuracy). But the current situation on this website is much different, and is more like the AI is learning with, following, or copying user IDs (whether correct or incorrect user IDs), in a sequential (vs. one-time) process. It remains unclear if this is the most accurate/best approach, especially if the alternatives haven't been tested. But in the event that this current AI system process will remain, regardless of what it's called, it should definitely be more clearly indicated in the guidelines at least that this is an AI-identifier or AI/identifier learning system, not a mere AI guess on it's own (except maybe for the first observer ID guess, when a user doesn't make their own non-AI ID).

Maybe reach out to staff directly with your questions / thoughts or share on the forum. It's possible somebody has answers to your questions or thoughts similar to yours and they may not see it in this comment section.

@dysm I deleted my forum account about a year ago on my own choice, not long before my curator status was removed, which you had a provenly (if minor) unfair role in. To clarify why I tagged you in this current post, it was because I was aware that you were in conflicts in past years with users over why your shared account was uploading numerous blurry photos. You vigorously defended the right for your account to upload them, to the present, as far as I know. You once in an old comment seemed to assume I agreed with you. What I'd actually say is that blurry, distant, or otherwise difficult to ID photos are unideal to upload, despite that they still may be "allowed" (meaning, critics could be wrong at least per current policy to say your account "can't" do that).

But it doesn't seem to be very difficult to work on improving photos so they aren't blurry. For example, you could help improve photo quality in photos taken on your shared account, and even mobile phones are increasingly improving their tools to prevent blur. But, my reason for saying "fyi dysm" on this page was to point out that tiwane had said something contrary to your view and your viewpoint of current policy, that the forum is evidently now place to discuss "bad photos," which some people could even take to mean that it's okay to call some photos "bad" (which some curators usually flag on observation pages). In other words, I was pointing out a seeming discrepancy between admin practices/policy/understanding.

Re: alexshepard, the one previous time I did send him a direct message he did reply and wasn't rude, unlike the typical silent treatment or bullying I get from some of the other admin/staff. As I think you remember, and as I've recently also said to @insectobserver123, I will typically reply/discuss things with you that are relevant to site policy or changes (e.g., in any flags, or in discussions about changing parts of the site), but no longer typically do/will do so in ordinardy contexts, because I consider your role in me losing curator status to have been unfair and dishonest, and similar (but not identical) to the role of some of insectobserver123's comments and moderation actions against me taken later (yet, he wasn't a curator when I lost curator status, so wasn't directly related then). Yet, I'd potentially consider at least partly changing my standards for when/where/who to discuss things with if past wrongs were to be corrected.

I'm not confused as to why you tagged me

Okay, but your first response was only about the different things I commented, so it wasn't clear whether you understood why I tagged you.

Maybe reach out to staff directly with your questions

Staff wrote this post, but again give silent treatment to only certain users including me, which they don't give to other users (i.e., depsite that my comments were somewhat lengthy (because the matters are technical), if someone else asked them, the staff would reply to them here). My questions if summarized are actually very easy to understand and answer, but probably won't be. As for the separate question about "don't call CV AI," I guess I'll just assume that the staff quietly deleted that from the guidelines without noting it, then or now. I'm fine to conclude discussing here. By the way, the forum also isn't a good place for such discussions, because certain staff, devs, or their friends not only frequently give multiple users' questions silent treatment there, but at times criticize or even bully certain users for contributing what are actually thoughtful comments. By the way, silent treatment, which is widely practiced by certain admin but only toward few users (so most users don't notice or experience it) is often defined as a form of bullying.

@insectobserver123

I just want to clarify that not only was I not a curator when bdagley's curator status was removed, I was completely inactive on iNat for a year or 2, and it was in this period of my inactivity that bdagley lost curator status, when I become active again I was disappointed to see this and had to search through old flags to figure out how it happened

I don't think I did suggest that you were involved in me losing it. Yet, since you say you were disappointed when learning of it, I would appreciate if you (would or had) said that more publicly or considered signing my petition to regain it. I'll also note that your recent accusatory judgements of me in hiding comments, and your earlier resolving of some flags without fairly indicating that I didn't violate guidelines, could only act against me regaining curator status. Since you also recently asked about the latter, I'll send you a private message about these things soon.

As for the CV AI question, I have seen everyone call it CV and occasionally get confused when I called it AI, and it is called CV in all iNat documentation, but I have never been aware of any guidelines that tell us not to call it AI, it's not in the community guidelines, not in the curator guide, and it seems like a strange thing to make a rule about at all.

I've instead seen numerous users, mostly outside of North America, have no idea what CV or computer vision is referring to. Also, as I did say I above, I distinctively remember the guidelines (or some guidance somewhere, possibly on a journal post about CV, or on the forum) saying something like "please don't call this AI," or at least, "please only refer to this as CV" (which would mean not to call it AI. The staff edit (update) Help documents as well, so something not being in it now doesn't mean it never was. If staff/devs (ones who know for sure what all the other staff and devs once said or wrote) could confirm that nothing like that was ever said, I'd grant that I had a false memory of it. Yet, that would also be a very strange and unlikely thing for me to have a false memory about if no one ever said it. I distinctly seem to remember it coming up when John or I would call CV AI at times (he typically always called/calls it "the AI"), probably mostly around 2021-2022.

I partly agree with @dysm that this post should now mostly resume if at all in discussing the CV/AI. Yet, in past similar CV update posts, users have asked questions and had discussions about how the CV currently works, and any suggestions or feedback. (which staff or devs actually replied to). For that reason, my original CV-based comments were on-topic, it's just that they got silent treatment. dysm mentioned the forum, but I deleted my account a year ago because the discussions were overly censored and certain users were mistreated by mods there (links to proof are available, if wanted). This leaves users like me, and the numerous users who never created a forum account little space to discuss website matters at all aside from posts like this, which in part explains why I did add discussions here. One other thing I'd point out is that these admin journal/blog posts always start out with a positive message (e.g., the recent "we estimate RG accuracy is 95%!), which can unfairly make anyone who then expresses a slightly differing view, or mere questions, as seeming to be acting "negative" to readers of the post. For that reason, I'd prefer if some admin journal/blog posts like this (but not this exact one) were phrased in their title as more of a discussion or "what do you think?"

E.g., a title like "We estimate that RG accuracy is 95%, feel free to discuss what this means to you or how accurate the estimation is." In actuality, the sample size was too small, the validator threshold was too low, and there were additional design limitations or flaws, so we can confidently say that we don't know what the RG accuracy percent is but that it must be overestimated. Because we know that some (of the numerous) global wildlife groups aren't checked/rechecked nearly as much as others by experienced (or any) users. But for me to say this could sound "negative," given the title implied it was 95% and that that was "good," but it isn't actually negative. It's just me (and others) providing constructive feedback on the design, results, and interpretation of results of a scientific experiment (which is encouraged, in science).

From this alone (there must be many more examples), the fact that nearly all staff, mods, and some curators, i.e., many admin (but not so much for many users and some other curators) mostly only recommend users refer to it as CV, and call CV the "preferred term," would naturally make many users feel like they're "supposed" to call it CV, not AI. Staff have also referred to it as things (something like) "the iNat Computer Vision Model," i.e., seeming to try to coin a new name from it and avoid the mention of the term AI. For completeness, some of the comments above (some of which I did quote in full) do claim the reason is that the CV "isn't really AI" or that it's more like machine learning (ML). Yet, many standard definitions, also including in some user comments I didn’t quote, define both the technical terms CV and ML as forms of AI, and of course, there can be multiple varieties thereof, and they can interact with human intelligence/learning to varying degrees (AI in a broad or multiple senses).

Furthermore, many or maybe even most published online or print articles about inat that aren't written by inat refer to its novel "AI” tool, not CV. One caveat (or contradiction) is that even the inat admin who remind users that "the preferred term is CV" have at (rare) times also referred to it as AI, and more recently asked users what it should be named. One thing seems clear: there seems to be an avoidance of using the term AI, which does have negative connotations to some people, but doesn't seem like a justified reason to discourage users from using the term. I'd describe the overall discussion over the preferred terminology as discouraging users from calling the CV AI, which I disagree with. Especially since the preference is mostly coming from admin including staff, that would lead many non-admin users to feel that they've said the wrong thing to use the term AI.

But, I infer from my experience identifying alongside taxonomists of cryptic wildlife groups like bees and wasps that there's more to this than AI being claimed by admin (but not by all users or admin) to be an inaccurate description. For e.g., the main issue for many identifiers is that this CV/AI continues to often guess inaccurately, especially on the first observer’s ID. Half a year ago I discovered and corrected that at least half of the European eumenine potter wasp obs. were misidentified at the genus level, but even after correcting them the CV/AI doesn't seem to improve noticeably. So, partly a speculative inference, but I assume that some experienced identifiers are unhappy that a (at least partial) non-human AI could outcompete their skills, but more so that it’s simply remains too inaccurate for acceptable RG standards, at least for many cryptic groups (for which it’s probably less than 50% accurate).

Anyway, none of that really matters very much, although can have pros and cons (e.g., the amount of time it takes people to correct wrong AI IDs), but I’d like to personally put it into effect now myself, if it isn’t in effect already, that everyone can call the system AI, CV, or ML, and that users and especially admin not discourage the term AI or continue to call CV the preferred term. And certainly not change people's post title's to remove the word AI. Lastly, the AI (and the two-ID RG system) are the two largest complaints Bug Guide editors have about inat, although I informed them that those things are unlikely to ever change, so are better to try to just get used to and overcome with their own IDs (but which does take a long time). I also believe that maybe eventually the AI will become more accurate even for cryptic groups, but years from now, and so I also somewhat disagree with how it’s often touted as having advanced, when in many of our day to day identifying it’s often still incorrect or even way off. Although, I agree that it’s good that more species are being added to it and that work is being done to improve it.