Looks like you have a few projects running
. Well… you can use 4’x8’ compressed Styrofoam sheet (maybe aluminum foil faced for additional insulating performance), cut it to the door size (make sure to leave enough clearance for the door jamb) and attach it to the surface with some appropriate adhesive, silicone, screws, etc.
Just make sure that those sheets can be left exposed, some are designed for installations underneath drywall, plywood, paneling or other finishing material. Additionally apply weather stripping along the door edges.
You can use blanket insulation wrapped in plastic and staple it to the surface. There are sheets specifically designed for garage doors insulation which I’m sure can be adopted for regular doors.
Depending on the opening size of the upper section of your staircase you could also install a hinged hatch (pre-cut piece of plywood) and insulate that with Styrofoam or blanket insulation.