Jim Fisher was an outstanding member of the men's golf team from 1954-57, serving as team captain in 1956. During his career, the Badgers were 34-8 in match play, knocking off many of the bigger schools in the region including Tulane, Ole Miss, Mississippi State and Southern Miss. He led the Badgers in key wins during the 1956 season at Mississippi State and at Tulane in back-to-back matches. After a very successful career on The Hill, Fisher return to his native New York where he was Metro Golf Association Amateur Champion in 1966 and won the 1966 Ike MGA Stroke Play Championship. Fisher also teamed with famous golf swing coach Butch Harman to win the 1963 Metro British Victory Tournament. Fisher was a longtime member of Shinnecock Hills Golf Club.