Officers with the Rescue 1 and Ladder 71 units will have to relocate after their firehouse was deemed "unsafe" by city engineers.
The conditions inside the historic building have been deteriorating for years and this past winter made the situation worse, officials say.
A Yonkers fire chief says there's a concern that bricks or even large sections of the building's walls could collapse onto the street. Barry McGoey, head of the Yonkers firefighters union, says scaffolding will be put up to protect the public.
It is not clear at this time if the building will be renovated or demolished.
"It's an unhappy day for the members of this fire company because the fear is that this firehouse will be condemned and never replaced," McGoey said. "And we want to make sure that if it is condemned that it will be replaced."
Both crews are being temporarily moved from their station on Saturday along with their equipment and personal belongings.