In english it's pronounced as Siva + Ganga = Sivaganga only.
In Tamil, Ganga is called as Gangai, so Sivagangai.
So as per government records in english, it is Sivaganga & in Tamil, it is Sivagangai.
Central government north indian officers (especially ITBP BATALLION & CISF BATTALION) used to call as "Shiv Ganga District" only.