Can FIR filters be unstable?
Stability: As FIRs do not use previous output values to compute their present output, i.e. they have no feedback, they can never become unstable for any type of input signal, which is gives them a distinct advantage over IIR filters.
Which type of filter is used in optimal transform design?
Infinite impulse response (IIR) filter design.