In June 1955, Rong Gaotang, the then vice president and secretary-general of the COC, stated at the Third Meeting between the IOC's Executive Board and national Olympic committees that the inclusion of a Taiwan sports organization in the IOC was illegal and that they should withdraw its recognition. However, the then IOC President Avery Brundage turned down the demand, saying that sport has nothing to do with politics.

In 1956, some senior IOC officials continued to raise the "Two Chinas" problem and insisted that Taiwan solely participate in the 16th Olympics in Melbourne. The COC lodged a strong protest with the IOC and withdrew from the games.

On August 19, 1958, the All-China Sports Federation announced a stop to all relations with the IOC. The COC and the sports associations affiliated to it withdrew from 15 international organizations one after another during June-August. The then IOC member Dong Shouyi also resigned.

The IOC then recognized a sports organization in Taiwan as a national Olympic Committee and selected its local official Xu Heng as an IOC member in 1970.

During this period, athletes from Taiwan Province participated in five summer Olympic Games, of whom Yang Chuanguang became the first Chinese Olympic medal-winner when he grabbed the runner-up in the men's decathlon events at the Rome Olympics in 1960.

Ji Zheng was the first Chinese woman to get a medal after ranking third in the women's 80m hurdles at the Mexico Olympics in 1968.