A proportion of the £90m cost of the scheme will be funded by the government from a record-breaking fine being handed to Network Rail this week.
The firm looks after much of the track, signalling and stations across Britain.
It is being punished by the Office of Rail Regulation for missing key punctuality targets on its long-distance services over five years.
Critics have long questioned the point of fining Network Rail for poor service, because in the end it is being stripped of cash that could be used to improve the lines.
Ten times faster
Now ministers say they will make sure the cash will be used to help people get online.
The new service, which could be 10 times faster than the service available at the moment, should be available within three to four years.
Network Rail says commuters will in future be able to get a connection through equipment installed alongside the track, rather than the current system of having to find a satellite signal as they go along.
Train firms will have to pitch for a slice of the money, but the government has indicated it will favour the busier commuter lines first of all - the most crowded 30% of the network, which carries 70% of the passengers.
The fine is being imposed for the late running of long distance trains on routes including Cross Country, East Coast, First Hull Trains, First TransPennine Express, Grand Central, Virgin, some First Great Western, East Midlands Trains and Greater Anglia.
Network Rail could face a potential penalty of about £70m based on the number of targets it has failed to meet. However, the BBC understands the fine will be substantially lower than this because the regulator has made an allowance for recent bad weather.
Source: BBC News
