Byard was a spectacular running back for Risley High School in the early 1950s. In leading the Tigers to the Georgia Interscholastic Association state championship in 1950, Byard tallied an incredible state-record 50 touchdowns including a whopping 10 TDs in a single game. A versatile athlete, Byard was a member of three GIA state championship teams that same year in three different sports – football, basketball, and track and field. The basketball team averaged nearly 70 points a game. At the state track meet, the Tigers clinched their third state title as Byard won the 100-yard dash, the 220, the 440, and anchored the mile and 440-yard relay teams. Combining speed with elusiveness and power, Byard then went on to an outstanding college football career at Morris Brown College in Atlanta, where he earned All-America honors in 1954-55. Also an avid golfer, Milton was featured in JET magazine in October 1953, being recognized as a cross-handed golfer who could hit the ball a very long distance. He was inducted into the Morris Brown College track hall of fame in 1980. Byard was then selected to the Georgia Sports Hall of Fame in 2009.