One of the reasons why buildings based on fertile soil feel so underwhelming on higher difficulties is that you rarely have the chance to find the soil early enough. Picking any of those buildings early feels like a huge gamble, yet you need a large infrastructure solely geared towars farm-based food compared to, say, meat. Farming buildings come online at the start of your second year at the earliest and you need to wait until Clearance to gain any rewards. This is delayed even further if you don't open new glades during your first year to avoid hostility. A lot of the higher difficulties usually reward tall play where you only open up very few glades, so the chance of finding fertile soil is further diminished.
An easy solution would be to allow fertile soil to spawn in your starting glade, so you could decide on farms knowing that you'll get some use out of them as soon as possible and can plan your infrastructure accordingly.