Firstly, It's a special train, so mostly it's schedule is set in such a way that it makes operations easier for the railways. Most specials are run to convenient schedules for the railways than for passengers. (This is expected as they need to manage route capacity).
Secondly, pretty much all trains take a very long time for the GTL-DMM run. In the reverse direction (DMM-GTL) the schedules are much faster. This is because the zonal handover slack for most trains is in the GTL-DMM section, before the handover to SWR from SCR.
Also...
more... SWR has to manage a lot of trains coming towards Bangalore in the morning. They would prefer a later handover for this special train, so that they can avoid any potential delays for their regular trains.