Just to clarify, challenges were never intended to be the end-all-be-all of trove flux and resource gains in Trove. The fact that they grant so many penta souls and trove flux was an oversight that has resulted in accelerated progression and mass inflation. These challenge changes are aimed at fixing that by making challenges more about attaining dragons (by making overall dragon soul acquisition much higher and requiring fewer caches) much faster/easier, and making trove flux/eye/soul gain much less prominent (by severely reducing or removing them from the caches).
This has some major implications, of course:
Freedom. Challenges will no longer overshadow all other activities in Trove, because their trove flux rates will be about the same as every other activity. You will no longer feel obligated to quit your Shadow Arena (Tower) team just to complete that hour's challenges, or drop what you are doing (be it building, fishing, mining, adventuring, etc.) just to complete the challenge, because you won't be missing out nearly as much. You will complete challenges to get dragons, and the extra trove flux/eyes will just be a bonus for spending the time to do so.
Realistic Progression. Currently, a player that has reached level 30 mastery can reach the "end" of progression in a matter of days or even hours if they try to do so. This was never intended, as reaching fully maxed out radiant gear was meant to be a very long-term and expensive goal. Such gear should feel "epic" when you finally finish a piece, but due to the current state of challenges, they simply feel like another rung of the progression ladder that you can easily climb. Progression will now be a grand journey once more, rather than the sole focus of the game which you can blitz through almost immediately.
Reduced Inflation. While you may feel that you will be losing out because challenges no longer throw flux at you, you may be surprised to find that you will likely have even -more- buying power with your flux after the update than before. Currently we suffer from inflation (meaning prices are high and trove flux is worth very little, due to there being far more trove flux than people can use/need), but ideally this update will help reduce inflation by reducing the amount of trove flux constantly entering the economy (and removing trove flux from the economy by fixed forging costs and possibly Shadow Lock costs for club worlds.) This will make it so that, even if you earn less flux per hour, your flux itself is worth more than before.
Now, I realize that this became quite the wall of text, so I apologize in advance for that! Hopefully it is not too much of an eye sore to read through! I'll provide a brief TL;DR summary below.
The challenge changes will make it so that we can enjoy all aspects of Trove equally while still remaining viable with our time spent, make progression feel less stressful/rushed and more epic, and help reduce the mass inflation that is plaguing Trove.
The Challenge Changes are intended to improve the game as a whole and help return balance to Trove by fixing a major set of issues and flaws that resulted from the initial incarnation of Challenges.
It will be difficult for some - especially newer players - to break free from their old mentality that Challenges were end-game and the sole goal/purpose of Trove at first, but I am confident that the majority of players will see very quickly that these changes were very necessary and a positive shift for Trove's direction!
To further phrase your point (and my original point on inflation) with numbers, flux has effectively inflated by 700% since the release of challenges (based primarily upon credit:flux rates and rare item rates.) Costumes that generally hovered around 5-8k flux now cost approximately 35-40k flux (though there is variation depending upon popularity.) Mounts that once cost 6-10k flux, now cost 45-70k flux.
Forging was meant to go up significantly in cost, as was evident by the huge cost barriers in attaining shadow 5 and radiant items (compared to the values of all other items and sources of flux income.) However, challenges pulled the costs of everything else in the game up to the new forging costs, which nullified the original intent of expanding progression through two progressively more difficult tiers of gear levels and greatly weakened flux in value. By weakening the value of flux, all other sources of flux were rendered obsolete; the only way to remain competitive with flux was to gain mass amounts of it (due to its diminished value), and the only source that could produce so much flux was challenges.
Had the Developers intended for flux to be so abundant, and forging to be so easy/fast, they would have increased the flux gains from the rest of the game world. The fact that challenges exploded the way that they did, is simply indicative of the fact that this was never the intention and that we were meant to handle shadow 5/radiant with similar flux rates to what we had originally. Avarem himself stated some time after challenges had been added that we were completing them much faster and with much more ease than they had anticipated; they likely expected challenges to be more difficult and less farmed, hence why the rewards entered the game at such high rates per cache.