Sorry for the necro, but I was able to fix this by simply disabling shader warming & lowering the lod scale from extra to high, though lod scale seems to reset itself so I'm not sure how much of an effect that has.
I'm running a healthy RX9070 with perfectly fine drivers if that matters to anyone. Hope the fix works for you guys too.