We hope everyone had a great start to 2024. Here are our January highlights! Our hidden theme for this week are files 🪰. If you missed last month's highlights, you can catch up here.

Species Discoveries

New species described
Listen to @johnmanning share the story of describing a new species of South African iris from a posting by @alexanderr.

🪰 This article describes a new species of Alaskan snakeworm described by @dssikes, @pereiratpl and colleagues and the role their iNaturalist project played in the discovery.

Lost species rediscovered
Re:Wild has been using iNaturalist as part of their effort to relocate lost species. This interesting article describes a recent publication summarizing their progress. So far, 27% of species on the list published in the study are represented by observations in their iNaturalist project.

Range Extensions and Distributions

Range Extensions

• This article describes @jggbrown, @kevinthiele and @mftasp’s discovery of the first known population of a rare violet from Tasmania.
• This study by @collinst, @dougvg, @frankiecoburn, and @acschneider documented the first disjunct Arizona population of a plant found in New Mexico.
• This study by @makario, @angel-romero, and colleagues used an observation by @antshrike69 to document the first known occurrence of a South American snake in Central America.

Improving Distributions
iNaturalist is increasingly used as the basis for studies to increase our basic understanding of species biogeography and diversity. For example, @k_jiaranaisakul and colleagues used iNaturalist records to fill out distribution maps for 14 species of lanternflies from Thailand. This month we highlight biogeography studies using iNaturalist data on:

• Plants from Uzbekistan
• Phloeidae Bugs from South America
• Arachnids from Cuba
• and Rays from Fiji

Invasive Species Science

A study published this month by @flo_grattarola and colleagues used 15 years of iNaturalist records to track the spread of invasive sea fig across Uruguay.

• There was another article on an invasive crayfish population in Canada and the role @drcrayfish played in identifying it.
• 🪰 @rodrigo_barahona_13 and colleagues studied the spread of a Woodlouse parasitizing fly in Chile with the help of an iNaturalist project

There were many other invasive species studies leveraging iNaturalist this month including studies on:

• The spread of Chinese Tallow and invasive moths in Florida
• An invasive beetle in South Africa
• Invasive plants in Russia in two studies here and here
• An invasive plant in Ukraine
• An invasive plant in Arkansas

Conservation and Monitoring

We were able to make our own case for how iNaturalist spurs conservation in a Revelator article this month.

As the southern hemisphere summer continues, we saw more stories this month from the New Zealand Government and about South African National Parks on using iNaturalist to help scale their limited monitoring capacity for conservation. The stories are connected with two of iNaturalist’s most prolific community members with the former profiling @jon_sullivan and the latter co authored by @tonyrebelo.

Protected Area management

• This study by @eric_tigrerito and colleagues assessed use of iNaturalist for protected area monitoring and management in Mexico.
• Similarly, this study by @lcmarinho1 and colleagues explored the use of iNaturalist for monitoring in a Brazilian State Park.
• This story about a The Nature Conservancy preserve in Michigan highlights the role iNaturalist plays in communicating the value of preserves to the public through showcasing the biodiversity that has been observed within them.

Restoration and Stewardship

• This article by @wncbotany describes a vision for regenerating landscapes for biodiversity and the role that iNaturalist plays.

Climate Change Science

iNaturalist is helping us understand how a warmer world is impacting biodiversity. This story describes winter bioblitzes that @ihz2 and colleagues are conducting to understand the effects of warmer winters in the finger lakes region.

This story features @jtanney0's work to understand the spread of sooty-bark disease to drought stricken trees in a warming Victoria through their project.

Warm water events off the Pacific Coast led to the spread seastar wasting disease. This story describes how @rbank and colleagues are using iNaturalist to monitor the spread of and recovery from this disease.

Many species undergo seasonal migrations that are complicated by climate change. This story describes the listing of Monarch butterflies in Canada as endangered and accompanying efforts to ramp up monitoring using iNaturalist by @carolyncallaghan and colleagues at the Canadian Wildlife Federation. This study used iNaturalist data to better understand the timing of Monarch migrations in California.

On the topic of climate change and species migrations, we enjoyed this article by @frances_figart about unexpected vagrants showing up in Great Smoky Mountains National Park.

Species interaction Science

iNaturalist images are valuable not just for documenting a single organism in space and time but also for documenting interactions between organisms and all sorts of other "secondary data" as described in this study. This month there were three great stories that used iNaturalist to better understand interactions between birds and food plants.

• The first was a study that used iNaturalist images to reveal a more diverse diet for the endangered Red-bellied Macaw than was previously known.
• The second story was a about the mysterious paralysis of lorikeets in Australia and the iNaturalist project that @david4262 and colleagues are using to determine which food plant may be poisoning the birds.
• Lastly, @emilymstone wrote a great article about her travels in Hawaii and the interaction between the endemic Nene goose and the endemic blueberry relatives they feed on.

iNaturalist Impact on iEcology and AI Research

This revolution that allows scientists to access species interactions and other secondary data contained in nature photographs like those described in the previous section is driven by the availability of large volumes of nature photography and new AI tools to efficiently pull out patterns from these images for analysis.

We’ve been calling this the Phenotypic Revolution, but from this study we learned a new term for it: iEcology (internet ecology). In the study, researchers used iNaturalist images to characterize the global shift in hermit crabs from using shells to using plastic trash as homes and the impacts on behavior and evolution.

Meanwhile,

• iNaturalist continues to be used as a standard benchmarking dataset for evaluating new AI and machine learning techniques such as those described in papers by Xiao and colleagues and by Kim and colleagues. These studies benefit iNaturalist by developing new concepts that we can integrate into our infrastructure as these experiments by @alexshepard show.
• Lastly, long time iNaturalist collaborator @gvanhorn and colleagues describe new research to combine Computer Vision and Large Language Models in this study.

iNaturalist data and model quality

Improving data quality is a very important piece of iNaturalist's strategy for increasing our impact. Accuracy as a key feature alongside ease-of-use evaluated in many reviews of identification apps such as three that came out this month here, here, and here. Likewise data accuracy is very important for iNaturalist’s use by the scientific community as explained in this study by @edulg. We are now complementing our monthly model updates with monthly data accuracy assessments allowing us to quickly learn and iterate on accuracy improvements.

iNaturalist’s Human Health and Social Science Impact

Participating in iNaturalist doesn’t just help species; it also helps humans. Here are three ways:

• Getting outside and connecting with nature even in winter is good for you as eloquently described by @marieviljoen in this article describing her winter iNaturalist finds. Likewise, this article has tips on how iNaturalist can help get your kids outdoors this winter.
• An increasing number of people are finding foraging as a gateway for connecting them to nature. This article provides some tips for how iNaturalist can help and this study uses an iNaturalist project to understand urban foraging.
• While some things in nature are healthy, others are unhealthy. This toxicology study uses iNaturalist to better understand stinging caterpillars in the United States. Likewise, this article mentions iNaturalist in the context of threats from venomous snakes in Australia.

🪰 This article explains how some flies, like the Australian blowfly, are harmful to agriculture while others, like the Australian hoverfly, are beneficial against a beautiful backdrop of fly photography from @acerad.

On the social science front, there was more work this month from @guiming and colleagues on understanding community patterns within the iNaturalist social network.

iNaturalist Events

As covered here, this month marked the 10 year anniversary of the Vermont Atlas of Life, one of the longest running projects on iNaturalist. Congratulations to @kpmcfarland and the entire Vermont Center for Ecostudies team!

January events included bioblitzes in the southern California mountains described here and a project by @johnslowry and colleagues to document biodiversity on Prince Edward Island covered here and here.

iNaturalist’s Education Impact

This profile on @cpavlisich describes using iNaturalist in STEM teaching activities at the Wolf Ridge Environmental Learning Center. Other uses of iNaturalist in education settings include:

• This story on documenting mushrooms on an Indonesian university campus.
• And this study on using iNaturalist for student activities related to the Mounds of Central Polissia in Ukraine.

iNatters in the News

During a stop on her adventures sailing around the world making iNaturalist observations, @mandapanda made such an impression snorkeling off Greece in December that she was covered by the local news. If you don’t speak Greek, we definitely recommend having your browser translate the article and give it a read. Other iNatters in the news this month included:

• This great video of @alan_rockefeller discussing mushroom photography and his excellent hat choice.
• 🪰 @sherylsr from Texas Parks and Wildlife wrote this excellent story on the wonder of flies!
• In Southern California, @ibhappy’s kestrel observation was featured as iNaturalist Spotting of the Month while this story highlight’s @gregwatson passion for desert reptiles.

Thank you to everyone who participated in iNaturalist this January and we look forward to a busy 2024 - your support makes it all possible!

Donate to iNaturalist

This is the twenty-second entry in an ongoing monthly (or almost monthly!) series profiling the amazing identifiers of iNaturalist.

The island of Madagascar is currently home to over 167k verifiable observations of over 10k species on iNaturalist, and David Rabehevitra (@davidrabehevitra) has added identifications to nearly 25k of them, making him the top identifier for this famously biodiverse place. 

Born and raised in Madagascar, David is currently studying the distribution of Tahina spectabilis, a Critically Endangered palm species, and became interested in nature from “being a scout and drawer/photographer from a young age.”

I loved traveling and living in the open air since I was a child, and loved growing things in my little garden. Since then my dream was to have a job that allows me to travel and discover as many places as possible in Madagascar, and around the world later on. When I got my bachelor degree I needed to decide which subject to study when entering university. I chose natural sciences first because it would at least get me close de nature, I then chose to study botany because I felt it would fulfill my dreams of traveling at least around Madagascar.

David joined RBG Kew Madagascar in 2016, where he first found out about iNat, and was later appointed as admin of the Zavamaniry Gasy (Plants of Madagascar) project. He tries to identify Madagascar plant observations on a daily basis.

I add identifications and comments on other users' observations for the pleasure of helping first: helping the platform to improve, helping and guiding others to go further in their research, helping the knowledge of Madagascar's flora and its conservation. I get the reward of a greater knowledge and capacity to recognize taxa from all over Madagascar!

I look at newly added observations, within which I scroll through all the available ones. Then I process from the most finely identified observations (species level) to the least (kingdom). I use The Flora of Madagascar, JSTOR, Plants of the World Online, the Tropicos Madagascar Project and my own database as resources when identifying.

In addition to sharpening his plant identification skills, David uses iNaturalist to gather distribution information for IUCN redlisting. 

The process of establishing The Red List of the Trees of Madagascar, in which I actively participated, used available data, including research grade observations from iNaturalist/Zavamaniry Gasy Project to realise the jump from 379 assessments in 2017 to the 3,000 plus assessments in 2021. I contributed to the identification of a relatively high number of observations to make them Research Grade and therefore usable in the IUCN process. I presented the evidence of this contribution of iNaturalist during a talk in the Madagascar ATBC session in 2019.

(Some quotes have been lightly edited for clarity.)

- when making observations of plants, David recommends taking “high visibility photographs and as much as needed! It requires the identifier to be in the shoes of the observer to make the identification as accurate as possible.” So try to get photos of leaves, flowers, and fruit, often from a few angles, as well as a wider shot showing the plant in full if possible.

- take a look at the most-faved plant observations in Madagascar!

- an observation of a rare Euphorbia plant in Madagascar by @fabienrahaingo, and identified by David, was iNaturalist’s Observation of the Week back in 2017!

Our Observation of the Week is the first Porcellio ombrionis posted to iNaturalist, seen in Spain by @amaneko!

Amanhuy Duque and Andrea Castro (who have the @amaneko account) are both biologists from the Canary Islands, and they specialize in terrestrial biodiversity and conservation. Amanhuy, who is quoted in this piece, tells me that he’s been fascinated by animals for as long as he can remember. 

My parents always took me for walks in nature and this helped raise awareness about its conservation and the importance of knowing how to look. From that early age it was clear to me that I wanted to be a biologist when I grew up, and that’s what I ended up doing. 

In Andrea's case, she always had an interest in nature and biology, but her true passion, birds, was discovered during her studies. She learned to mix it with her passion for drawing to make some incredible illustrations.

Last month, Amanhuy visited the island of El Hierro with his family.

I had cut myself on a piece of glass on my foot a few days before, so while they were finishing the walk I took the opportunity to search the area some interesting species to photograph. 

One of the species that caught my attention the most that I found on this walk was this Porcellio woodlouse that showed striking ornamentation. I took several photos trying to show the important characteristics for later determination and the rest was the work of the small group of researchers and isopod specialists who discussed the ID of this observation.

El Hierro is the second-smallest island in the archipelago, and it is both the one with the fewest number of observations on iNaturalist and one of the least studied at a scientific level. Just in two small trips to the island, including this one, a dozen new records have emerged (which will have to be verified in most cases with the collection of some individuals). 

I reached out to one of those identifiers, @pepe_rando, about his identification and this observation. Pepe tells me there are twenty-three known species of Porcellio in the Canary Islands, most of which are endemic. 

Porcellio ombrionis is only present (at least so far) in El Hierro and La Gomera. It was first described by Albert Vandel in 1954. It is characterized (to the naked eye) mainly by its granulations, which are distributed all over the body, and are quite developed. And by its head lobes (the three "horns" the animal has on the head), specially the central one, which in this species is quite developed and curved upwards. This last characteristic is what made me rethink my ID of the observation. I had originally thought that it was Porcellio studienstiftius, which is an extremely similar species, but smaller, and with a central lobe that is not as developed, but when I took a second look at the second picture, it is fairly clear that this specimen's central lobe is way too developed for it to be Porcellio studienstiftius. In general, both species are characteristic enough to automatically rule out any other species from the islands, so it had to be one of those two.

Oniscideans are an often overlooked group. Maybe this observation is only relevant to a few of us who are addicted to looking at pictures of these creatures for hours trying to figure out exactly what species exactly they are, but I like to think it will be useful for anyone who wants to look for the species in the future, be it to study it, or just for pure pleasure (as is the case with me), and can now do so through these pictures. This observation is (most likely) the first picture ever taken of this species. Maybe seeing pictures of these unique species, rather than some drawings from seventy-five years ago, will spark interest in protecting them, seeing as arthropods are usually underrepresented when it comes to conservation efforts. Many of the Oniscidea species from the Canary Islands still have no pictures that can be found online, but just recently, thanks to iNaturalist, we now have pictures for 8 species (1 Ctenorillo and 7 Porcellio).

Amanhuy and Andrea joined iNaturalist just over two years ago.

We discovered iNaturalist a bit by chance, when finishing our master’s degrees. We started by identifying some photos that Andrea had taken over the years and soon I was hooked and ended up buying a camera so I could take photos too. iNaturalist has managed to increase the love we have for nature and, especially in my case, it has made it our obsession to search, investigate and photograph the different species we have on our islands. 

We owe much of the knowledge about the species we have in our territory, especially arthropods, to iNaturalist and the wonderful people who participate in this platform. I have spent hours reading scientific articles and interpreting keys just to try to identify some of the species we have photographed. We are generalists in terms of the species we photograph, although we have more interest in fauna than flora. Andrea has a special interest in birds, while I have found small life both in the sea and on land fascinating, with a special weakness for nudibranchs and the fly families Asilidae and Tephritidae, as well as lace bugs.

(Some quotes have been lightly edited for clarity. Photo of Andrea is above, Amanhuy below.)

- some iNat users are part of the American Isopod and Myriapod Group (AIMG), which has some excellent tips for photographing isopods for identification! Check out their project.

- take a look at the First Known Photographs of Living Specimens project on iNaturalist!

We released a new computer vision model today. It has 84,878 taxa up from 83,622. This new model (v2.11) was trained on data exported on December 31, 2023.

Here's a graph of the models release schedule since early 2022 (segments extend from data export date to model release date) and how the number of species included in each model has increased over time.

Here is a sample of new species added to v2.11:

• Animalia
• Mollusca
• Amphibia
• Protozoa
• Fungi
• Chromista
• Arachnida
• Insecta
• Actinopterygii
• Aves
• Reptilia
• Plantae
• Mammalia

Our Observation of the Week is the first Megamerus alvarengai leaf beetle posted to iNaturalist! Seen in Brazil by @alenilson.

“As a child I had a lot of contact with nature through my grandmother, who was a farmer,” recalls Alenilson Rodrigues. 

I sometimes accompanied her in her efforts in the field and that's where my passion came from. As time went by I started going to the forests as a way to de-stress and I started taking photographs at the suggestion of my wife. At that moment I was having personal problems that were distracting me and photography came to me as a way to change the direction of my life. Without a shadow of a doubt my wife always deserves to be highlighted for my photography. If it weren't for her suggestion we wouldn't have these images today.

I developed a special taste for photographing arthropods, especially flies and moths, and I ended up developing a technique that generates images with a white background, giving me the nickname “studio macro”. It's a very laborious method because I only register live animals and after registration I release them in a safe place, most of which I find on my property or nearby.

He came across this Megamerus alvarengai beetle while at home, and at first didn’t know it was a particularly special find. 

There is no great story, I simply believe that any living being should be photographed so that there is the possibility of knowing its occurrence as well as a means of protecting it, as I believe that only what is known is respected. While watering some plants I noticed the beetle hidden among the leaves and registered it. It was @borisb who identified both the being and its importance.

Boris Büche (@borisb) is iNaturalist’s top identifier of beetles, having added IDs to over 650k (!) verifiable observations of them, so I reached out to him about Alenilson’s find to get his thoughts on it. He says that the subfamily to which it belongs (Sagrinae) is an ancient branch of leaf beetle evolution. Most extant members of it are found in the old world.

When I saw the discovery by @alenilson, my first belief was that it would be something completely unknown. I remember well my identification of Atalasis sagroides four years ago, then found to be the single American representative in the basic checklist by Blackwelder (1945/46).

M. alvarengai was discovered five years afterwards in the state of Rio Grande do Norte, Brazil (seven examples, found by Moacyr Alvarenga in 1950, scientific description in 1956 by Francisco de Asis Monrós). The beetle was found again only in 2009-2013 during a trap sampling campaign in Caatinga formations in the state of Paraíba (twenty examples, in xerophilous vegetation) - and now, by Alenilson.

Alenilson found two of these beetles (here’s the other), and both were on coconut palms, but Boris thinks it’s likely not their host plant. Not much is known about the species, but host plants for other members of the subfamily are dicots, not monocots. 

Larvae of its Malagasy relatives form galls (unpublished observation), and pupae of two Australian forms (including a Megamerus) have been found within earthen cocoons underground. One iNaturalist observation of a female laying eggs would support the suspicion that larvae of Megamerus are root miners.

“I use iNaturalist to promote the animals I find in my region,” says Alenilson (above). 

Here I can meet experts as well as see  the occurrence of different animals in different points of interest. After getting to know the platform I discovered people and places, and of course, several animals that occur here in the state along with other people who share the same interest.

(Some quotes have been lightly edited for clarity and flow.)

- you can follow Alenilson on Instagram and YouTube!

- this observation is the First Known Photographs of Living Specimens project!

This is the twenty-third entry in an ongoing monthly (or almost monthly!) series profiling the amazing identifiers of iNaturalist.

I met Sofia Zvolanek and her mother Jenny at an iNaturalist meet-up arranged by @thebeachcomber in Sydney back in 2022 and was really impressed by her passion for nature - especially fungi. She’s currently the top identifier of verifiable fungus observations in Australia with about 80k identifications made, so I thought I’d take the opportunity to profile her here.

Currently a 23 year old graduate from Sydney, Sofia’s been living in a suburb by the bush for about 18 years, and tells me

I think I've always been interested in nature - the colours and patterns and shapes and minute details, interacting with things and watching how they behave, seeking out interesting creatures under logs, collecting cool shells and other objects from the beach. I've been picking up bugs since I learned to walk. My interests were nurtured by my Mum - as a kid I spent a lot of time with her gardening, bushwalking, and doing crafts. I also spent a lot of time outdoors up trees or in the bushes.

Her first memories of mushrooms involve her staining the walls of her cubby by rubbing Coprinellus everywhere (“artistic spirit I guess”), but she really got into fungi over the past few years, her interest sparked by friends who brought them some foraged mushrooms.  “I discovered that mushrooms could actually taste nice,” she says. “From there I gained an interest in identification for foraging purposes, which grew into identification for curiosity's sake after I started using iNaturalist and the information became much more accessible to me.”

Sofia got hooked on iNat in 2020, when she had to make five observations for a university class. 

…and from there I did not stop. Then the lockdowns hit, and I was spending a lot of time at home or in the local bush, walking for fun or to travel to uni once classes were back on, while the mushroom season was in full swing. The ephemeral and mysterious nature of mushrooms, along with their interesting and varied appearances, makes them particularly appealing to my novelty-seeking brain, and I decided to answer the endless stream of “Oooh what's that?” by photographing and posting them all to iNaturalist. From there I juggled classes and assignments while spending as much time as possible on identification sprees or cram-studying the variety in fungal morphology by scrolling through the iNaturalist observed species lists order by order or family by family (for example) to get an idea of what exists and how it's arranged in the tree of life. I've learned pretty much everything I know about broader fungal taxonomy through iNaturalist and its users.

Funny story - I spent my first few weeks on iNaturalist under the mistaken impression that @thebeachcomber was one of our course instructors, because he was so diligent about welcoming my classmates and I to the platform.

And she gives back by adding a lot of identifications, both on her phone when she has a break or is in transit and via the iNat website. Her current strengths are Agaricomycetes, Xylariales and Lecanoromycetes (mostly Cladoniaceae), as well as a few slime molds (Mycetozoa).

Most of the time I rely on my memory of patterns observed while cram-studying the observed species lists, but when a species or taxa piques my interest I'll go on research binges to dig up as many relevant taxonomy papers and descriptions as I can find in a few hours to gain a better understanding of morphology and species delineations. Unfortunately most of the Australian compiled resources are years out of date, with old or misapplied names. I'm also not subscribed to any of the journals, so I tend to just put the taxon into Google Scholar and hope for the best.

These tend to be followed by an hour or more of scanning through the observations of the most relevant overarching taxonomic group to apply the newfound IDs to unidentified observations. I've also recently been making use of the “Similar Taxa” tab to help resolve the frequent mix-ups between similar-looking but often widely taxonomically distributed genera of earthstars and coralloid fungi, along with some other groups.

And why add all those IDs? “Sorting things and solving problems is satisfying and helps contribute to global knowledge of fungi,” she explains. “Every so often I learn something new or notice something scientifically interesting. Sharing my interests with others is fun and I enjoy people's excitement when they learn something new. Finding friends and a community, and making connections. I feel like I'm doing something worthwhile with my time.”

Now that she’s graduated from university, Sofia’s current plans are to find a job and get settled into “adult life,” but she may return to school for a graduate degree in the future, hopefully focusing on fungal taxonomy or ecology. She also has a pipe dream of starting a mycological garden “to showcase the beauty of different fungal forms and give the general public a better understanding and appreciation of fungi.”

Need tips for making identifiable fungus observations? Sofia’s got some advice:

• Take multiple photos from multiple different angles and make sure they’re in focus. The most important is a wide shot showing the fungus in its habitat. Also, close-ups of the whole fruiting body from different angles (top, side, underside, etc), whether or not there is a stem and what it looks like. Acquiring a small mirror such as a compact or dental mirror will help capture those critical underside and stem angles if you want to leave the fungal fruiting body intact.
• Other useful things would be closeups of different structures and textures on the fruiting body hymenium structure (gills/pores/teeth/smoothness) and attachment to the stem/substrate, as well as the base of the stem and any other lumps, bumps, tufts or shreds on the fruiting body.
• Breaking or bruising can cause some fungi to react to air, which can help with identification, and you can also make spore prints (only a thumb-sized part of the hymenium is needed). 

Some of Sofia’s favorite fungi taxa: Bird's nest fungi (Nidulariaceae), Cannonball fungi (Sphaerobolus), Crown-tipped coral fungi (Artomyces), Pretty-lips (Calostoma), and regular Coral fungi (Ramaria). Taphrina cornu-cervae is my favourite plant pathogen. Psathyrella aquatica is average looking, but pretty cool because it’s (I think) the only known agaric that fruits under water. There are many other aquatic fungi, but most of them are single-celled or have insignificant fruiting bodies. Rosecomb deformities (for example https://www.inaturalist.org/observations/164406178) are pretty wacky looking.

You can see Sofia (and her mum!) at about nine seconds in on this interview video with Thomas Mesaglio!

Today we’ve launched our second Observation Accuracy Experiment (v0.2). Thanks to everyone helping us conduct these experiments. We're learning a lot about iNaturalist observation accuracy and how to improve it.

Changes in this Experiment

We made two changes to the experimental design from v0.1 based on feedback:

• We changed the validator criteria to be at least 3 improving identifications of the taxon from the same continent since many reported not feeling comfortable identifying taxa outside of their regions of expertise.
• We messaged candidate validators rather than emailed them since many reported not noticing emails. We also only left a 4 day interval (rather than 2 weeks) between contacting validators and the deadline since last time most validating happened within the first couple days after contacting candidate validators.

Eventually, we’d like to increase the sample size from 1,000 to 10,000, but we’re sticking with 1,000 until we get a few more kinks out of the methods. The page for this experiment is already live and the stats will update once a day until the validator deadline at the end of the month, but you won’t be able to drill into the bars to see the sample observations or the validators until the deadline has passed.

New Data Quality Assessment condition for photos unrelated to a single subject

We also made one change to iNaturalist functionality in response to findings from the study. We added a new “Evidence related to a single subject” condition to the Data Quality Assessment table to make it easier to remove observations with multiple photographs of unrelated subjects from the verifiable pool.

Two of the incorrect Research Grade observations in Experiment v0.1 were of this type which we estimate to be ~350k observations in the entire iNaturalist dataset. The norm up until now to make these observations casual has been to set an identification to the nearest taxonomic node shared by the multiple subjects and then vote no to “Based on the evidence can the Community Taxon still be confirmed or improved?”, but many found this process to be clunky and confusing. We hope this new Data Quality Assessment condition will make it easier for the community to remove these observations from the verifiable pool where they negatively impact data quality and distort features on iNaturalist (such computer vision model training and the browse photo tool) that assume observation photos all relate a single labeled subject.

Thank you!

Thank you to everyone contacted as a candidate validator for participating in this experiment. We expect that considering location may decrease the percentage of samples validated compared to the previous experiment by constraining the candidates available to validate, so we very much appreciate your participation in helping get as much of this sample validated by the end of the month as possible. As always, please share any feedback or thoughts you may have on the topic of data quality. We’re excited to continue learning from these experiments and your feedback about data quality on iNaturalist and what changes we can make to improve it!

Results (added 2/29/2024)

Thanks everyone for participating in this 2nd experiment. The validator deadline has now passed, meaning that on the experiment page the stats will no longer update, the validators are now visible, and the “Accuracy results by subset” bar graphs are now clickable allowing you to drill in to see the observations behind the graphs.

In this second experiment, we estimated the accuracy of the iNaturalist Research Grade observation dataset to be 97% correct and the accuracy of the Needs ID subset to be 79% correct. The graph below shows the first experiment (v0.1) in lighter bars and this experiment (v0.2) in darker bars. The results are very close which is reassuring.

These estimates of average accuracy of the entire Research Grade observation dataset are in line with our expectations, largely because the iNaturalist dataset is skewed towards a relatively small number of common, easy to identify species (e.g. mallards, monarchs, etc.) that have an outsized impact on the average. Nonetheless, we wanted to touch on three sources of uncertainty in these estimates: validator skill, sample size, and the uncertain category.

Validator skill

We are assuming that candidate validators can perfectly validate observations as Correct, Incorrect, or Uncertain. We know this assumption is not exactly correct because there are a small fraction of situations where more than one validator looked at the same observation and they disagreed (e.g. validator 1 says Taricha torosa and validator 2 says Taricha granulosa OR validator 2 says Taricha because you can’t rule out Taricha granulosa). This happened on 1.6% of the time in v0.1 and 1.2% of the time in v0.2. This error might be higher if we are underestimating disagreements because validations aren’t done blind (i.e. validators can see each other’s validations). But the error might also be lower because each observation was validated an average of 4 times, so assuming the validations are mostly independent even if one validator made a mistake it was reviewed an average of 3 more times. In future experiments, we’ll do more work to estimate uncertainty in the labels (Correct, Incorrect, or Uncertain) stemming from imperfect validator skill. But while this uncertainty stemming from validator skill is non-zero, it’s likely close to 0. Furthermore, there’s no reason to assume that this uncertainty would bias towards inflating the accuracy by overestimating the proportion correct, it could just as well bias towards underestimating the proportion correct.

Sample size

Because being correct or not is like a coin-flip, we can put confidence intervals on our estimates of average accuracy based on the sample size. As the sample size increases the confidence intervals become narrower. We can compare our estimates and confidence intervals from v0.1 to our estimates and 95% confidence intervals if we pool v0.1 and v0.2 together effectively doubling our sample sizes.

We already have a large enough sample size (n) to have fairly confident estimates for large subsets of the iNaturalist database such as the Research Grade (RG) Accuracy Estimate. After v0.1 (n=534) our estimate and 95% confidence interval was 0.95 (0.93 - 0.96) and pooling v0.1 and v0.2 (n=1109) it is now 0.96 (0.94 - 0.97).

However, for smaller subsets such as RG Fungi our confidence intervals are still quite large. After v0.1 (n=6) our estimate was 0.83 (0.36 - 1.00) and pooling v0.1 and v0.2 (n=19) it is still 0.95 (0.74 - 1.00) - so somewhere between 74 and 100% accurate.

For very small subsets, our sample size is still much too small to provide useful estimates. For example, to estimate the accuracy of RG Rare (taxa with fewer than 1000 observations) African Insects our estimate even after pooling v0.1 and v0.2 (n=5) is 0.6 (0.15 - 0.95). For other subsets (e.g. RG Very Rare (<100 obs) African Insects) we have a sample size of zero and can’t make any estimate.

The size of our sample is adequate for getting relatively confident estimates of average accuracy for the entire iNaturalist database and for large subsets (e.g. the RG subset, the North American subset, the Insect subset etc.), but these sample sizes are too small to yield confident estimates for more niche subsets (e.g. RG South American Fungi from 2022, etc.). We are very interested in variation in average accuracies across these subsets and look forward to growing the sample size to the point where we can better understand this variation.

Uncertain Category

Ideally, we’d be able to label all observations as Correct or Incorrect. But because we don’t have the capacity to get validations on all observations in the sample, some remain Uncertain. This was 3% of the RG subset in each of v0.1 and v0.2. Since we are calculating Accuracy as the percent correct (as opposed to 1 minus the percent incorrect) this Uncertain category is biasing us to underestimate the Accuracy. The true average accuracy of the iNaturalist Research Grade observation dataset is somewhere from 0% to 3% higher than our estimates because of this bias which is on par with the uncertainty interval resulting from sample size.

Thank you and next steps

Thanks again for all your help with another successful experiment. We’re amazed by the capacity of this incredible community to validate these samples. We hope you’ll click on the graphs and explore the results here. We plan to run another experiment at the end of March. We may try keeping the methods the same and increasing the sample size from 1000 to 10,000. Or we may make another change to the methods such as more changes validator candidate criteria. Thanks again for making these experiments possible!

Our Observation of the Week is this school of Golden Cownose Rays (Rhinoptera steindachneri, Raya tecolote in Spanish), seen in Mexico by @flore_boituzat!

Flore Boituzat grew up in the French countryside and says she’s always been interested in nature. She’s a food engineer, and tells me “I had the opportunity to do one semester in Mexico to do marine biology, that's where I started to dive and go snorkeling.”

She also met her boyfriend Alberto Alcalá (@alboertoalcala) while there, who’s a diver and an oceanologist, who introduced her to iNaturalist. 

After her semester in Mexico, Flore had to finish her last year of schooling in France but she recently returned to Mexico. “As a welcome gift, Alberto took me out for a dive where we saw all these beautiful rays. I felt really blessed. :)”

Golden cownose rays occur in the eastern Pacific Ocean, from Peru up through Mexico, and are often found in groups like this. Like other rays, it doesn’t have teeth but bony plates in its mouth for crushing its prey - mostly bottom dwelling species. It’s listed as Near Threatened by the IUCN, as it’s often caught as bycatch by artisanal gillnet and trawling fisheries. 

Flore (above) continues to use iNat, and continues to share her nature photos on iNaturalist. 

I like nature photography and he knows about the biology part, so thanks to iNaturalist I learn about the species I'm seeing, and I can also see where a species is found a well as, it’s habits and characteristics, so it's easier for me to find and to capture photos of it. I think biodiversity is part of the beauty and the magic of this planet, and to take pictures of biodiversity helps me share it; I find it very nice and fun to see what's surrounding us.

(Some quotes have been lightly edited for clarity. Photo of Flore was taken by Alberto Alcalá.)

- here’s footage of a golden cownose ray school!