That looks like you have it set for cycling. It defaults to a “Safe” mode which would likely take you on alternate paths that are less travelled by cars. At the bottom you can change the route option to “Road” or “Balanced”. If it still wants you to take a dirt road after selecting “Road”, then it’s a problem with OpenStreetMap.

You can fix those issues by going to openstreetmap.org then zoom in on the area and edit the map. Make sure that the main road exists all the way through and that the dirt road is tagged with highway=track.