If you have a JR Pass, can you take a bus from Hiroshima Station to the Castle (you can also pay). This goes every 15 minutes or so between 09.00 and 17.00. Its not that hard to find and its on the North side of the station.
There are three museum in Hiroshima city.
Hiroshima Museum of Art : 2mins from Hirosima Castle to the museum by foot
Hiroshima Prefecture Art museum : 7mins from the castle to the museum by foot
Hiroshima Modern Art museum : 30 mins by bus or take taxi.
And which museum?
Get quick answers from Hiroshima Castle staff and past visitors.