The owner of a pub-bistro says it has had to cut its hours due to fewer lunchtime customers.