Jacket with fleece insert