Los Blancos were expected to close the gap on Real Sociedad and Atletico Madrid at the top of La Liga to just three points but suffered a shock reverse at the Alfredo Di Stefano.
An early penalty from Lucas saw them fall behind before a Thibaut Courtois error allowed Joselu to double the guests’ advantage. Casemiro got one back late on, but Alaves had their chances to score more.
“We made our worst start of the season and we couldn’t change the dynamics of the game. That bothers and worries me,” Zidane admitted after the match.
“In the second half we fell short. We did very little to change the course of the game.”
Madrid have won only seven of 14 fixtures this season and it is the lack of consistency that is frustrating the boss.
“I don't have an explanation for it. And for tonight either,” he admitted, having seen his side beat Inter in the Champions League away from home in midweek.
“We alternate good moments with bad moments. When they score a goal after three minutes, things get very complicated for you.
“The other day in Milan we play a good game and today ... It is reality, our reality today. What we have to do is keep working, recover players and change the dynamics. We don't have consistency.”
Zidane, meanwhile, lamented the injury situation and offered an update on Eden Hazard’s fitness after he was forced off in the first half.
“We’ve got a problem with regularity,” he admitted. “We’ve got a lot of issues with injuries. It’s not an excuse, but it is reality.
“I think and hope Hazard is just a hit. I hope it’s not muscular.”
Madrid are back in action again on Tuesday, when they face Shakhtar Donetsk away in the Champions League.
Having lost the previous meeting between the clubs, they will see avoiding defeat as essential.