I have had the same over a period of a week, also a couple public holidays. I did do a few things during the week though and that skews it even more. A solution for this might be to be able to set "planned work time", so if we have PTO we might set this day(s) as 0. It would then also allow setting weekends or otherwise with planned work time. Overall you would have an additional data point where actual work could be compared with planned work.

How it resolves the average work times being displayed? Maybe there is a configuration that allows user to set a threshold where if the planned work time in a day < 8 hours than exclude from the average calc for day, if weekly planned work time < 40 hours then exclude from the average calc for week.

I faced the same because I had two days off in the same week.