Pop the door out and you'll need to try and get some lining inside the cavity. Anything you put in there will decrease the clearance between the door and cavity unit so check your door slides 'straight' into the cavity. Sometimes the door or the cavity might have a twist or warp which will limit clearance.
Line it with 3mm ply or corflute (the stuff For Sale signs are made of). I'd stick it in with liquid nails. Use anyhting that is thin and flat. You don't care too much about the unsulation properties, you just want to stop a draft. Obviously you can stick it in in sections, the slider units normally have about 3 or 4 horizontal rails in them. You may only need to do one side depending on where the draft is coming from.