Upcoming Changes to Lists

The the new life list tool we released today is completely separate from the existing list functionality on iNaturalist and the original entity known as a life list (from here on “original life list” to avoid confusion). Here, we describe how these existing lists work and some changes we’re planning to improve scalability as iNaturalist grows.

Part of the reason why the existing list functionality on iNaturalist is so complicated is that lists are used for many different things. For example, there are normal user lists, original life lists, traditional project lists, default place check lists, and other place check lists. We’ll describe these different kinds of lists in turn and how they act differently. But let’s start with the most straightforward and simplest kind: the normal user list.

Normal User Lists
A great example of a normal user list is the favorite taxa list you can display on your profile. To do this, you create a list called “Favorites” and add some taxa to it.

The taxa added to a list are stored in records called “listed taxa.” Each listed taxon has a description and comments and keeps track of “observation stats” which include the number of times you’ve observed the species and links to the first and last observation. The latter allows you to do things like filter the listed taxa that haven’t been observed.

Normal user lists are in many ways analogous to Guides on iNaturalist. They serve as ways for a user to create and maintain a list of taxa that (aside from the observation stats) don’t interact with observations.

Original Life Lists

When you create a normal user list, you have the option to “Make it a life list.” Checking this box means that the list will create listed taxa for any observation you’ve made. If you restrict to a higher-level taxon and/or a place (e.g. “Brazil Frogs”), it will only create listed taxa from observations of that taxon in that place.

You can still manually create and destroy listed taxa on original life lists independently of listed taxa being automatically created from your observations. And issues with original life lists being out of sync with observations has been a persistent point of confusion. Let's call this automatic listed taxa creation from observations functionality “auto-listing functionality” for short.

Traditional Project Lists

Traditional projects have a “must be on list” rule which is really just a shortcut for many “must be in taxon” rules. In fact, in the newer collection projects, we no longer bother with these lists and just allow users to add many “include taxa” project rules. Nonetheless, we still support traditional projects and thus still support traditional project lists to facilitate this ‘must be on list’ rule. They behave exactly like normal user lists (i.e. no auto-listing functionality) except that the observation stats are filled from observations in the project rather than observations made by the user.

Default Place Check Lists

When you create a place, you have the option to mark “check lists allowed”. Doing so will create a default check list for the place. Place check lists behave very much like original life lists in that they include auto-listing functionality from research grade observations made within the place. Likewise, people can manually create and destroy listed taxa independent of the observations made in the place. The observation stats are filled by observations made within the place.

Listed taxa on place check lists also store “establishment means” (e.g. native/introduced) for species in that place which is used throughout the site. Likewise, the listed taxa on standard place default checklists also determine the "presence places" in an atlas. The existence of an atlas for a species disables auto-listing functionality for that species on default checklists.

Other Place Check lists

In addition to the default place check list, places can have other check lists. They can be restricted to a higher-level taxon and they can be marked as “comprehensive” from the list edit page to indicate that all species in that higher-level taxon for the place are included in the list. Like the default place checklist, they also have auto-listing functionality.

Problems with Lists and Proposed changes

The two major problems with lists are that functionality to track observation stats and auto-listing functionality aren’t scaling well, meaning that as iNaturalist continues to grow the server requests this functionality generates are bogging down the performance of the site.

We’ve reviewed lists on iNaturalist and have determined that they basically serve two separate use cases:

1. The ability to view a set of observations in species list form

2. The ability to maintain a reference list of species (independent from observations) to add context to and compare with observations (e.g. establishment means, atlases, etc.)

We think we can better serve use case 1 with new dynamic tools viewing observations in list form like the new Life List tool we’re unveiling today. We plan to build an analogous new Place Check List tool to view the species observed in a place in list form. This will allow us to remove the problematic auto-listing functionality from all lists so that they can be more focused on serving use case 2: acting as an independent reference list to add context to observations.

We also plan to remove observation stats from listed taxa. We realize that this will remove functionality to compare what species on a list have been observed and which haven’t been observed - this is, for example, what controls the color (green means observed and yellow means unobserved) of the check list places on taxon maps. But, if there is demand for this kind of functionality to compare a list with a set of observations, we think we can build more scalable functionality to do so rather than the existing observation stats.

To be clear, we're not planning to remove any existing listed taxa. We are only planning to disable the auto-listing functionality and the functionality that maintains observations stats on listed taxa.

Here’s our proposed roll out of these changes:

Phase 1 (today):
Launch new dynamic life list tool

Phase 2 (next few weeks):
Remove original life lists (they will become normal user lists)
Remove observation stats from normal user lists and traditional project lists

Phase 3 (sometime in the next few months):
Launch new dynamic place check list tool

Phase 4 (sometime in the next few months):
Remove auto-listing functionality and observation stats from place check lists
Remove other place check lists (they will become normal user lists, leaving only a single optional default place check list for each place)

Posted by loarie loarie, October 27, 2020 21:17

Comments

Thumb

Thanks Scott. Can you explain more about "We also plan to remove observation stats from listed taxa" or provide examples of what it looks like now to have observation stats from listed taxa, and what will look different after the change?

Posted by muir 28 days ago (Flag)
Thumb

Hi muir, in the second image above the orange arrow is pointing to the display of 'observation stats' on a listed taxon page. e.g. for American Pika on the California Checklist the listed taxon would be https://www.inaturalist.org/listed_taxa/5443 and the observation stats store the number of RG obs (441), the First iNat observation ("August 15, 2007 in Evolution Lake, Kings Canyon National Park, CA by gdurkee") and the Last observation ("October 17, 2020 in Twin Bridges, CA 95735, USA by paul31").

As described in the post, the scale of processing to keep these observation stats updated is a growing problem, but they support functionality to:
1) search for listed taxa on a list that haven't been observed, e.g. Mammals on the California Checklist that haven't been observed https://www.inaturalist.org/check_lists/312-California-Check-List?view=photo&taxon=40151&observed=f
2) color the listed taxa that display on taxon maps green (RG obs exists) or orange (no RG obs exists)

So if we removed observation stats they would no longer be displayed on listed taxa, we'd remove the 'observed=yes,no' filter from list searches, and the listed taxa would all display one color on taxon maps (e.g. just green rather than green and orange)

Posted by loarie 28 days ago (Flag)
Thumb

Got it (I think). I like the orange-shaded places on the map, but it won't be a big loss in my opinion.

Posted by muir 28 days ago (Flag)
Thumb

@loarie Considering how often (or not) taxon maps like the above are accessed, I wonder if it would be an unacceptable performance hit to simply have taxon maps start accessing two place checklists (the new dynamic and old original ones) to determine map coloration, instead of the one being accessed now, for each map place? Color everything in the original checklists orange, then change places to green that are in the new dynamic checklist? (Or something equivalent but more computationally efficient). Or put another way, would doubling the "draw" time of a map be unacceptable?

Posted by jdmore 28 days ago (Flag)
Thumb

Traditional Project Lists :

This is a much cooler way of adding a list than the current collections project where one has to add (or remove) species one by one. For long lists and many lists (e.g. https://www.inaturalist.org/projects/nemba-alien-species-south-africa and https://www.inaturalist.org/projects/south-african-red-list-plants-and-animals) managing them is a pain.
Every time the Red List is updated or the legislation on alien plants is promulgated (once or twice annually) the long lists have to be edited one by one and they are in a meaningless order wasting a lot of time searching for them. Adding the functionality of Traditional Project Lists to current Collections Projects would be a great help. The alternative for Collections projects is a batch add and batch remove functionality, but why not just use the Traditional Project Lists for this?

Traditional Project Lists surely do not need auto-listing functionality - they are just lists for use as filters in projects (traditional and ?why not? collections).
Can they not serve the same role for places?
And could they not also be an option within Filters in ID and Explore tools?

Posted by tonyrebelo 28 days ago (Flag)
Thumb

How does the "plan to remove observation stats from listed taxa" affect the Taxon (Species) Page.
e.g. https://www.inaturalist.org/taxa/132848-Protea-cynaroides
It has a "Last Observation" box, which is most useful.

I am going to miss the Last Observed entry in the Place Lists: .e.g.
https://www.inaturalist.org/check_lists/63740-City-of-Cape-Town-Check-List
and especially the missing taxa (incl. subspecies)
https://www.inaturalist.org/check_lists/63740-City-of-Cape-Town-Check-List?q=&view=photo&observed=f&commit=Filter

We used that extensively for planning our City Nature Challenge priorities.

But so long as there is an equivalent to the Unobserved Species in the Life List - for the Place List that uses the various checklists as the filter (not the global All-Life filter) - then that will be fine.

Posted by tonyrebelo 28 days ago (Flag)
Thumb
Posted by tonyrebelo 28 days ago (Flag)
Thumb

@tonyrebelo Thanks for pointing that out. I have moved it.

Posted by andrewgillespie 28 days ago (Flag)
Thumb

Accessing Lists.

Can I please request that ALL Lists are accessioned in the Lists menu (https://www.inaturalist.org/lists/). Currently, it unnecessarily involved (impossible) to find to which reserves (places) one has added a “tree” or “aliens” or "grass" list to.
This should include not only:

Life Lists
Normal User Lists (== user lists)

But also:

Traditional Project Lists (== filter lists),
Other Place Check Lists (== checklists)

And what about a tab for "Lists" in the "Search" results

Posted by tonyrebelo 27 days ago (Flag)

Add a Comment

Sign In or Sign Up to add comments