"I've had to get rid of one member of staff already. Other actions will swiftly follow," he told journalists in a briefing before delivering his budget speech.
Earlier this week his department head, Thami Manyathi, was suspended by premier Helen Zille pending the outcome of a probe into alleged tender irregularities.
Among the deals under scrutiny is a R91 million transport contract, which was not put out for tender before being awarded to Games Transportation Systems Services.
Carlisle said he was unhappy about firing staff when it appeared they had been put under a great deal of pressure to do things they should not have done.
"But I want to say this: when we have got this bedded down, and when the investigations are in a further state, we want to go looking for the people who instigated these matters," he said.
The probe would look both within the political hierarchy, and outside it at the private sector.
"Our aim certainly would be to put as many of them in jail as we could," Carlisle said.
"We need to make it clear to South Africans that when you do these kind of things, there's a price to pay."
In his speech, Carlisle said the department had spent more than R1.5 billion on consultants in the past three years while service delivery took a back seat.
"There have been other bizarre and sometimes corrupt actions," he said.
Instead of going on the building of new schools and houses, his department's money had "leaked into the pockets of already rich and deeply cynical people".
In the briefing, Carlisle confirmed that construction of a new hospital at Mitchell's Plain on the Cape Flats would be delayed.
The province had seen revenue collection figures coming out of National Treasury, and they were "pretty hair-raising".
"VAT, company tax, are falling through the floor," he said.
He very much doubted whether the Western Cape or any other province was going to get as much money next year as planned in the medium term expenditure framework.
"I'll eat my hat if we get that money next year," he said.
"Money is going to be extraordinarily tight."
Sapa
The ANC has no interest in porn; the JSC is impartial; and the SABC wants public thanks...
Should politicians be entitled to excessive luxury? Ebrahim Moolla takes on Rebekah Kendal...
Do you need a laugh? Then check out our offbeat stories from around the world...