We released a new computer vision model today. It has 83,622 taxa up from 82,023. This new model (v2.10) was trained on data exported on November 26th.
Here's a graph of the models release schedule since early 2022 (segments extend from data export date to model release date) and how the number of species included in each model has increased over time.
Here is a sample of new species added to v2.10:
Our Observation of the Week is this ice-encrusted Artomyces cristatus fungus, seen in the United States by @orofeaiel!
Originally from the US state of Georgia, Jen F moved west to Washington state “specifically for the rain, mountains, and better job prospects in the field I'm in (healthcare),” and took up hiking as a hobby, motivated by the beautiful landscape there.
Hiking and being outdoors, I've discovered, is my greatest source of joy, peace, and healing. Though I've been hiking for many years, I've only recently begun to learn to identify species and it has made my time outdoors so much more fun and engaging and I feel even more connected to the land and life having a greater understanding of it. My favorite species to find and identify are fungi and slime molds.
So, after hearing that a friend had found some Artomyces cristatus fungi not too long ago, they went out together to find them. “I'm always on the lookout to ‘acquire’ new species sightings,” she says, “[and] after some hunting we were able to locate some. As we photographed them, we were observed closely by a curious goat and horse pair of best friends from a nearby property.”
A type of coral fungus (the tips of the fungus point upward), Artomyces fungi grow on rotting wood, as you can see in Jen’s photos, and look pretty great with some ice crystals on their fruiting bodies. They’re in the order Russulales, a highly diverse group of mushrooms that come in quite a few shapes and structures.
Jen (above, in a self portrait) tells me she grew up in a family that wasn’t particularly outdoorsy so she didn't get into nature until adulthood. “I use iNaturalist to catalogue my sightings, see what others are observing in my area, track observations of my favorite species, and learn about new species I want to see, she explains.
Using iNaturalist has definitely changed the way I see and interact with the world in a major way. I look more closely and go more slowly when outdoors, observing life I had not noticed in my years before learning about species. I'm much more in awe and appreciative of the world around me.
Jane Goodall worded it more aptly than I ever could: “All the time I was getting closer to animals and nature, and as a result, closer to myself and more in tune with the spiritual power that I felt all around. For those who have experienced the joy of being alone with nature there is really little need to say more; for those who have not, no words of mine can ever describe the powerful, almost mystical knowledge of beauty and eternity that come, suddenly, and all unexpected.”
- take a gander at the most-faved Russulales observations on iNaturalist!
- six years ago @bouteloua's icy coneflower was an Observation of the Week!
Now, you can see common names in multiple languages!
iNaturalist has been translated into 57 languages, but many people want to use the platform in one language while learning common names in another.
We’ve recently added functionality to address this, allowing you to experience iNaturalist with multilingual common names. For instance, the image to the right displays the Mayan Name Ta alongside the Spanish name Chamiso for an observation of Viguiera dentata on the iNaturalist iPhone app. (Viguiera dentata is also known as known as Tahonal and Tajonal in Spanish and Taj and Sak xo´ xiiw in Mayan)
This feature aims to facilitate knowledge sharing and learning on the platform. In the Yucatan peninsula, Maria Cecilia Alvarez (@naturalista_cecilia) has been working with university students in citizen science and environmental education. Proficiency in both Spanish and Mayan names opens up broader opportunities to engage with and share knowledge with local Mayan communities about indigenous species.
The image below features @jhovannyflota and @yessi_yam, two students from the Regional Technological University of the South, Yucatán, observing Ta on their campus. The Yucatan Peninsula is one of the most important regions for the production of honey and Ta is an emblematic pollinator species for beekeeping. Ta's antimicrobial properties also makes it an important medicinal plant used by indigenous people across Mexico.
To use this feature, go to your account settings (A) and navigate to the Content & Display section (B). From there you can select lexicons (C), add them (D), and control how they are displayed alongside Common names in the same language as your locale. Your locale determines the language that the whole site appears in and is set in the Account section or from the footer of the website. You must configure multiple common names from the website but once you do, they will display on the iNaturalist Android and iPhone mobile apps. Additional detail is available on the iNaturalist Forum.
One of iNaturalist's core goals is generating high-quality biodiversity data to advance science and conservation. We are launching some experiments to better understand the accuracy of these data. Here’s how they will work:
We draw a random sample of observations from the iNaturalist database of observations.
We choose potential validators and distribute the sample among them, considering their past activity identifying observations on iNaturalist (more details in the FAQ below). We assign the same observation to multiple validators to increase the odds that a large fraction of the sample will be reviewed.
We send emails to each validator with a link to their subsample loaded in the iNaturalist identify tool, instructions to identify each as best they can, and a deadline after which we will use the new identifications to assess the accuracy of the sample.
The instructions are for validators to add the finest identification they can to each observation. We’ve included the instructions in the FAQ below if you’re curious about the details. We know this means that some observations that are already Research Grade might get a flurry of redundant confirming identifications.
The top level statistic we are aiming to estimate is Accuracy (the percent of the sample that is correctly identified). We will do this by assuming that the new identifications added by validators are accurate and comparing them to the observation taxon (more details in the FAQ below) to classify observations as correct, incorrect, or uncertain. We use these classifications to calculate high and low estimates of accuracy.
If the sample size is large enough, we may be able to understand variation in accuracy by dividing up the sample by geography, taxonomic group, quality (research grade etc.), and other characteristics.
We’ll be piloting this protocol with our first experiment later this month (Experiment 1). We’ve already generated the sample (Step 1) and selected potential validators (Step 2). We plan on emailing the potential validators on January 17th (Step 3) with a deadline of January 31 to give validators two weeks to identify their subsamples (Step 4) before we share the results the first week of February (Step 5).
For this first experiment we generated a modest sized sample of just 1,000 observations. We distributed it among 1,219 potential validators attempting to assign each observation to at least 5 validators in order to increase the chance that the observation will be reviewed. Here are some characteristics of the observations in the sample from Experiment 1:
Thank you so much in advance if we contact you as a potential validator and you choose to participate. We couldn’t do this experiment without your help and expertise!
How exactly are you selecting potential validators?
If an identifier had made at least 3 improving identifications on a taxon, we considered them qualified to validate that taxon. Improving identifications are the first suggestion of a taxon that the community subsequently agrees with.
For example, Identifier 1 adds an ID of Butterflies to an observation. If Identifier 2 later adds a leading ID of Red Admiral, Identifier 1's ID on Butterflies becomes an improving ID. If Identifier 3 later adds a supporting ID to Red Admiral, Identifier 2's ID in Red Admiral becomes an improving ID.
Note, we count both Identifiers 1 and 2 as having 1 improving ID on Butterflies (since Red Admiral is within Butterflies). Only Identifier 2 has an improving ID on Red Admiral.
How will I know if I was selected to be a validator?
You’ll receive an email from iNaturalist titled “Will you help us estimate the accuracy of iNaturalist observations?”
How large are the samples you’re sending to validators?
It varies. For Experiment 1, many validators are only being sent a single observation. No validator is being sent more than 100 observations.
What if I can’t identify an observation in my sample
Please add the finest identification you can add based on the evidence in the observation. Even if it’s ‘Birds’ or even ‘Life’ that’s ok. We won’t learn anything from non-disagreeing identifications that are coarser than the observation taxon but that’s ok. The only thing that will really hurt our assumptions is if you add an incorrect identification.
What if an observation in my subset has no photo or there are other issues like missing locations?
We've excluded observations without media (photos or sound) or with the no votes on the "Evidence of organism" data quality flag from subsamples. Observations with other data quality flags like missing locations may be included. Please do your best to identify them despite the issues.
I don’t want to add confirming identifications to observations that are already research grade.
We realize that this can be undesirable - e.g. some identifiers like to preserve their reputation of not “piling on” etc. But we need new identifications on all observations to estimate accuracy so we appreciate if your help with the assessment by adding a new identification in these cases. If it helps, feel free to mention that you’re participating in this experiment and linking to this blog post in your identification remarks.
What happens if multiple validators of the same observation give different results?
If the multiple validators all agree (i.e. if their identifications are of the same taxon or one is a coarser non-disagreement to another), we will choose the finest taxon as the “correct” answer. If multiple validators disagree (i.e. if their identifications are on different branches or one is a coarser disagreement to another) we will investigate these conflicts on a case-by-case basis to decide how to proceed. We’re hoping these conflicts will be rare.
What if I’m 100% sure of the genus but only 90% sure of the species? Which should I identify it as?
Unfortunately, there’s a degree of subjectivity involved in an identifier's particular comfort level. For example, some identifiers need to be able to see specific characters to feel confident enough to add a specific identification. Others are more comfortable using things like location as constraint (e.g. “I can’t positively rule out other members of this genus based on the photo but there’s only one species that occurs here”). Please identify as fine as you feel comfortable adding, but if you need a rule of thumb, add the finest identification that you think is 99.99% correct. In other words, if you think there’s less than a 0.01% chance that the out of range look-alike could have hitchhiked to the location then it's fine to choose the in range species even if you can’t see the diagnostic character.
What assumptions are you making when you estimate accuracy from this experiment?
We’re assuming that the sample is representative of the entire iNaturalist dataset and that validators do not add incorrect identifications. The larger the sample size, the stronger first assumption becomes. We've only selected validators with at least 3 improving identifications for the respective taxon, but that doesn't mean they never misidentify that taxon. We've added redundancy by attempting to select 5 validators for each sample, but some samples have no qualified validators and we know we wont' get a 100% response rate.
What happens if you don’t get much of the sample reviewed, either because no one participates or because no one can add correct identifications?
If we can’t get an observation in the sample reviewed or the validators can’t add an identification as fine as the previous observation taxon, we will code it as "Uncertain". We aren’t making any assumptions about uncertain observations, but they are increasing the uncertainty in our estimates. The worst case scenario is that we have so many Uncertain observations that the bounds on our accuracy estimates are too broad to be useful (e.g. accuracy estimates with a low of 50% and a high of 100%).
What if I’ve already identified an observation in my subsample?
Please review your old identification and if it is still relevant you can skip it. If you no longer think that your older identification is correct, please add a new identification.
What if your sample size isn’t large enough to get robust estimates
That’s possible. We won’t know until we get a sense for how much participation we get, the portion of Uncertain observations we’re left with, and how the community responds to this pilot. If the response is good, we can increase the sample size in future experiments. This will likely be necessary if we want robust estimates for under-represented regions and or taxa (e.g. African fish).
How will we get to see the results of the experiment?
We’ll post a report summarizing the results of the experiment to the iNaturalist blog a week after the experiment deadline. We'll comment here with a link when it's up.
What instructions are you sending to validators?
We’ve copied them below in case you’re curious (note: instructions will have sub samples tailored to each validator. The 10 here are just meant to serve as an example):
Dear loarie,
Will you help us with a study to estimate the accuracy of iNaturalist observations?
Here is the subset of 10 observations that we think you can identify based on your activity on iNat. Please add the finest ID you can to each of the observations before 2024-01-31.
We’ll calculate accuracy by comparing your ID to the Observation Taxon. You can skip observations where you’ve previously added an ID if that ID is still relevant. For more on how we will count agreements and disagreements, keep reading.
IDs equal to (A) or finer than (B) the Observation Taxon will be counted as Agreements.
IDs on different branches (C) or coarser than the Observation Taxon where you choose “No, but…” to the “Potential Disagreement” dialog (D) will be counted as Disagreements.
IDs coarser than the Observation Taxon where you choose “I don’t know but…” to the “Potential Disagreement” dialog (E) will be counted as Uncertain.
We’re so grateful for your help as an identifier on iNaturalist, and thank you very much for participating in this study. Please read this blog post to answer frequently asked questions about this experiment.
With gratitude,
The iNaturalist Team
Our Observation of the Week is this Yellow-winged Bat (Lavia frons), seen in Nigeria by @abubakaringim!
“Every December of each year during the last couple of years, my bosom friends and I make it a tradition to visit the Dagona Waterfowl Sanctuary in Yobe State, a section of the Chad Basin National Park,” explains Abubakar S. Ringim (who goes by Ringim), a conservation biologist, activist, and photographer from Nigeria. “It's a wonderful opportunity for us to unwind, have fun, and marvel at the breathtaking beauty of the waterfowl during their winter migration.”
Here's a picture of Ringim (on the right) with rangers at the sanctuary during this trip:
One day, the group ventured into the thicket, trying to get a better view of the birds at Oxbow Lake.
As we pushed our way through the dense bushes, we were met with sharp thorns that seemed to form an impenetrable barricade. But we persisted, walking and almost crawling to get a spot with a clear view. It was then that my dear friend Haroon drew my attention to the little bat - a yellow-winged bat perched low on the ground. We approached it with caution, not wanting to disturb it. After taking a few photographs, we moved on with our nature walk, exhilarated by the beauty of the bat.
The genus Lavia is in the family Megadermatidae, which are known as “false vampire bats.” They were once believed to feed on blood like actual vampire bats do, but actually feed on insects and other small animals. Yellow-winged bats, which measure about 58–80 mm (2.3–3.1 in) in length, are only known to eat insects, which they catch using a “sit and wait” method. They can be found across much of Sub-Saharan Africa and they prefer to roost near bodies of water.
Ringim (above, mapping biodiversity at the Yankari Game Reserve), credits his parents for “instilling in me a deep appreciation for the natural world through their love for nature, travel adventures, and wildlife documentaries.” He studied zoology in university, earned a master’s in biodiversity conservation, and his “current research interests are interdisciplinary, encompassing both the natural and social sciences, and include a wide range of areas such as birds, butterflies, wetlands, citizen science, and human dimensions of wildlife management, which is currently the focus of my PhD research inquiry.”
He’s been using iNat for about six years now, for both research and education.
Looking ahead, I'm excited to explore how the data we're collecting can inform biodiversity conservation action and policy decisions in Nigeria. As part of a dedicated team of citizen scientists, we are mapping Nigeria's biodiversity through multiple projects, including the Nigeria Biodiversity Mapping Project and the Baturia and Yankari Biodiversity Mapping Projects. Our efforts continue to yield fascinating discoveries, such as new wildlife additions to Nigeria's checklist, range extensions, leucistic wildlife species, and numerous first-time records of wildlife species captured by indigenous Nigerians, among others. I aspire to leverage the iNat data to propel conservation efforts in Nigeria.
Personally, iNat has changed the way I view and interact with the natural world, constantly reminding me of the diverse array of species that inhabit our planet and how our actions can affect them positively or negatively. More importantly, it reminds me of the fact that without the data of species' occurrence, abundance, and distribution in this ever-changing world, biodiversity conservation is far from becoming a reality, especially in a country like Nigeria. I am constantly motivated to venture out and capture photos of anything I come across!
(Human photo credits: Idris Muhammad Jalo (above) and Haruna Mohammed Abubakar (below))
- here's a video of a roosting yellow-winged bat.
- the Afribats project has nearly 10k observations of 235 species!
Our Observation of the Week is only the second Pancheria ouaiemensis plant posted to iNaturalist! Seen in New Caledonia by @narido.
Dorian.KC (“narido” is an anagram of Dorian), a primary school teacher from China, says his passion for plants started in his senior year of high school. “Honestly,” he tells me, “there aren’t any complicated reasons or exciting origins. I love things living and growing in their own way, in the most natural, the most comfortable, the most vital fashion.”
Dorian studied changes in flower color across different altitudes in Taiwan for graduate school, and has since traveled to many places to explore flora around the world.
Plants live in places different from me, lead a lifestyle different from me, communicate with creatures differently than I do, and witness stories and histories differently than I do. Once I have taken photos of plants in the wild, I feel like I can get a complete sense of everything about their lives. Not only digital records, but also the beautiful flowers they have, the pollinators they interact with, the waterfalls by which they live. I can easily connect with them when I take and enjoy the photos. It’s quite pleasant to develop such interests with iNaturalist.
A few months ago Dorian traveled to New Caledonia, extremely excited by its high number of endemic plants. He saw this Pancheria ouaiemensis while on a hike in the northern side of the island.
I climbed for almost 2 hours, taking photos non-stop of the plants along trail, and I was drenched in sweat. Unfortunately the stunning views were covered by heavy fog, but surprisingly I did finally come across a Pancheria ouaiemensis plant. Although I saw many species of Pancheria in southern New Caledonia, it’s the first Pancheria I recorded in the north. Because all Pancheria species are endemic to New Caledonia and they have gone through adaptive radiation, i.e., diversified rapidly into multiple new species, each Pancheria fits perfectly into their local environment.
Their leaves are thick and tough, with a waxy surface which literally shows me how Pancheria ouaiemensis has adapted to frequent wildfires, seasonal drought, and poor nutrients in the maquis habitats. In addition, the cream-white color and sweetly scented capitula could attract many types of pollinators - not only bees and flies, but also butterflies and beetles. This may allow Pancheria ouaiemensis to adapt to the declining pollinators on island habitats. Such an incredible book of amazing evolution, living vividly in front of me.
Dorian (above, on Mont Mou in New Caledonia) wonders at French botanist Jean Armand Isidore Pancher, the namesake of Pancheria. What was it like exploring a remote island without the kind of references we have available now?
Fortunately, iNaturalist provides us with a very friendly environment and a professional reference. When planning my trip, iNaturalist helped a lot to locate the regions and trails where I could find my target plant species. And I also search and compare my records using iNat databases, as many of the most recent images and records are here - which is extremely helpful. Then I love to share my findings with people around the world!
You don’t have to worry about having a DSLR or a professional camera - mobile phones are encouraged on iNaturalist and this indeed breaks down barriers for nature observation. It’s cool and agreeable to record, to learn, and to share nature using iNaturalist.
Right now, there are no excuses to not record nature.
(Some quotes have been lightly edited for clarity and flow.)
- take a look at all 200+ observations of Pancheria plants on iNaturalist!
- in 2020 @damienbr's observation of the Dumbéa River Pipefish from New Caledonia was the first one of that species in decades!
Our Observation of the Week is this trio of Orange-spined Drone Flies (Eristalis nemorum, Eristale interrompue in French), seen in France by @nicolashelitas!
Nicolas Helitas tells me he was raised by his grandparents in the southwest of France. His father, who lived near Paris, would spend a lot of time fishing when on holiday and took Nicolas with him. “I think the hours I spent quietly on the bank of ponds gave me the love of being in nature,” he says, which led him to study biology.
I made an herbarium, and I started to look more closely at plants - especially orchids. Those plants are often rare, so I started to take photos instead of collecting them. Then I looked at the butterflies which were visiting the plants, then dragonflies after that, while a passion for photography started to grow in me. Now my main interests are still botany and entomology, but I'm potentially interested in everything in nature!
I currently have more than 200k images on my hard drive. Even if lots of them are redundant (I don't like selecting and deleting!), that represents a lot of data. I offer some images to nature protection associations who look for illustrations, but most of those data have not been used. A friend of mine told me about iNaturalist some time after we had discovered GBIF. We had found an excellent way to give value to years of observations! The link between the two sites was the key for our interest in iNaturalist: the information is stored and shared together with all other scientific sources, and offered to researchers.
One of those photos, of course, is the subject of this blog post. In June of 2015, Nicolas ventured out in search of a rare butterfly. While he never did see the butterfly, he did spot drone flies behaving in a curious way. “I was looking closely at all the wildflowers along the trails, and I noticed those hoverflies on a daisy. I had never seen this interaction of 3 flies, and took a series of images to try to have the 3 in focus.”
I reached out @matthewvosper, the top iNat identifier of orange-spined drone flies, for some information about the taxon and this behavior. He told me that the genus Eristalis, which has been split reapeatedly since its original description in 1804, contains about 100 species and has about 50k more observations than any other hoverfly genus on iNat.
The larvae of Eristalis and similar genera live in muddy stagnant pools. They use their long breathing-tube tails like a snorkel, providing them with the rather delightful moniker “rat-tailed maggots”. They can be attracted to your garden by making simple “hoverfly lagoons”.
E. nemorum is one of the smaller species of Eristalis. It is characterised (though not uniquely) by its glassy clear wings with a very small wing stigma (the stigma is often important for identifying Eristalis). It's a very typical-looking Eristalis with clearly triangular orange spots near the front of the abdomen (which can be faded in females), and narrow white bands at the segment margins. It has a bit of a ginger-haired scutum, normally well dusted with a characteristic pattern. It has a holarctic distribution, and it is most similar to E. hirta and E. rossica (which also have short stigmas).
One of the most distinctive things about E. nemorum however, is the unique behaviour exemplified in this Observation of the Week. A female enjoying a flower will be “guarded” by a potential mate hovering directly above her. When the male finds her he touches her with his legs before assuming this position (apparently to confirm the identity - they have been observed touching small dark objects in this way and flying off). They can hover like this for many minutes. When the female leaves, the male follows her. This behaviour doesn't guarantee a mating opportunity however, as other males can get involved. Additional males hover in a towering stack of suitors, maintaining a constant distance from each other. It's not entirely clear how this situation gets resolved when the female flies off! This behaviour is a diagnostic character of E. nemorum.
Nicolas (above) continues to post his photos to iNaturalist and use it as a way to identify his subjects. “I haven't spent a lot of time exploring images from over the world,” he says, “but I'm planning a trip to Greece in April, and iNaturalist will be a nice tool to prepare for it.”
(Photo of Nicolas was taken by Elisabeth Gaillard. Some quotes have been lightly edited for clarity.)
- you can find Nicolas’s website here!
- check out our Identifier Profile on a group of fly identifiers who’ve made ID resources for hoverflies!
- here’s some footage of Orange-spined drone flies courting in Ireland.
- this is a nice paper describing courtship behavior for orange-spined drone flies. [PDF]
Thank you to everyone who participated in our first ever Observation Accuracy Experiment that we launched 2 weeks ago. From this experiment, we estimate the accuracy of Research Grade observations to be 95%. Keep in mind that these results are drawn from a relatively small sample size, but this is the first quantitative accuracy estimate we've had. You can explore all the results and click through to the sample here. We're very excited by these result and eager to build on them with more experiments!
As explained in the methods section, we generated a sample of 1,000 observations on January 16 and selected 1,232 candidate validators who each had at least 3 improving Identifications for the corresponding observation taxon. By today’s January 31 deadline, 887 (72%) of the candidate validators participated, validating 96% of the sample. On average each observation was validated by 4 validators. From these we calculated Accuracy (correct, incorrect and uncertain) and Precision.
For the Research Grade subset, 95% were Correct, 3% were Uncertain and 2% were Incorrect. The average Precision was 99%.
By clicking on other tabs you can see the Verifiable subset (Research Grade + Needs ID) had a lower accuracy of 90% Correct, 5% Uncertain and 5% Incorrect. The average Precision was 75% with fewer observations sitting at rank species compared to the Research Grade subset. For the entire sample (including Casual observations) the accuracy was 87% correct, 8% uncertain and 5% incorrect. The average Precision was 73%.
The "Accuracy Results by Subset" section allows you to see the results grouped in different ways, for example by Continent. These bars are clickable allowing you to for example see the 6.25% of Research Grade observations from Asia that were assessed to be incorrect. Remember that these characteristics such as a quality grade, continent etc. are from when the sample was generated and may have since changed (e.g. if someone marked what was a Research Grade observation as captive/cultivated thereby making it Casual).
The blue button allows you to toggle between frequency and percent to see the sample sizes involved. For example, 6.25% is equivalent to 3 incorrect Research Grade observations from Asia.
The results from these experiments are valuable in helping us all develop a shared understanding of what’s driving the accuracy and precision of the iNaturalist dataset and what changes we should make to improve it. We’re excited to continue doing experiments like this on a monthly basis. We want to learn from you what worked or didn’t work for you as part of the validator process, ways we could improve the experiments, and thoughts on opportunities suggested by these results to improve accuracy and precision.
Thank you so much for all your work creating this unique iNaturalist dataset by adding observations and identifications, and for your participation in making experiments like this possible!
We wanted to end this post by digging into the incorrect and uncertain Research Grade observations assessed by this experiment in more detail.
Of the 13 Research Grade observations that were incorrect, 8 were what we’re calling the Other Branch scenario in which validators selected a taxon on a different branch (2) than the observation taxon (1).
3 were what we’re calling the “Shouldn’t rule out Y” scenario in which validators thought the observation taxon (1) was too precise and that it should be rolled back by adding a disagreeing ID higher up on the tree (2) because other alternatives shouldn’t be ruled out.
2 were what we’re calling the “Multiple species” scenario where it's unclear what the subject of the observations is (in these case because multiple photos of different organisms were included in a single observation). The observation was Research Grade at the taxon shown in the first photo, but the norm in these scenarios is to roll the observation taxon back to the common ancestor of all the taxa associated with the various subjects.
Of the 14 Research Grade observations that validators classified as Uncertain, 3 were because we were unable to find or get responses for candidate validators.
7 were what we’re calling the “Uncertain beyond Z” scenario in which validators thought the observation taxon (1) could be correct but were uncertain about alternatives so added a non-disagreeing ID higher up on the tree (2).
4 were situations where more than one validation conflicted which we code as Uncertain. In 3 of these cases, one validator went with what we’re calling the “Likely X” scenario by adding an agreeing ID (2) to the observation taxon (1). The other validator went with the “Shouldn’t rule out Y” scenario described above.
In the 4th case, one validator went with “Likely Y” while another went with “Other Branch” by choosing a hybrid between the observation taxon and a sibling species. This shows that there is some subjectivity in validation, particularly in how precise observations should be.
Our rule of thumb is that validators should base their decision on avoiding risk by ~99% (e.g. if you think there’s a less that ~1% chance that it could be an alternative like Painted Lady or West Coast Lady you should select “Likely X”, if the probability of an alternative is greater than that you should select “Shouldn’t rule out Y”, and if you are uncertain about those probabilities you should select “Uncertain beyond Z”), but we realize that judging these probabilities will always be subjective.
We’re hoping that as we run more experiments, we can start developing shared archetypes for common situations that lead to incorrect or difficult-to-assess observations to help us design improvements to reduce them (e.g. the Multiple Species scenarios). We know only a handful of potential scenarios were represented in this small sample, and are eager to see what other common scenarios come up in future experiments.
