As current China has entered into the new normal, the great effort to advance the supply-front structural reform is the main content of future economic work. Under this background, how banking as the principal part of financial sectors supports the supply-front reform of the real economy to really achieve the efficiency improvement of financial services to the real economy deserves attention. This paper uses the data of Chinese industrial firms to discuss the promotion impact of banking market structure on the increase in industry TFP. It states that the development of small and medium-sized banks can perfect the credit allocation structure, then effectively reduce industry resources misallocation and increase industry productivity. On the other hand, the development of small and medium-sized banks not only can raise the survival risks of zombie companies and force them to exit the market, but also can alleviate credit constraints faced by high-efficiency firms when expanding the production and promote the growth of high-efficiency firms. Therefore, the optimization of banking market structure is beneficial to optimal allocation of credit resources, and is of great significance to a solution to zombie companies and the advancement of the supply-front structural reform.