It is true that many of those downloads came shortly after the game's launch. The creators were celebrating the 500 million mark in September 2016, or just weeks after its debut. Growth has clearly been much slower since -- it's just that the game managed to keep and foster its player base well after others gave it a pass.
The figure illustrates the challenge Harry Potter: Wizards Unite faces in gaining widespread adoption. There are legions of Pokémon Go players, and they might not be in a rush to take on a second game that plays like the one they already know. With that said, Pokémon's endurance shows precisely why you can't rule out success for other Niantic games, even if they don't quite become cultural phenomenons.