Entrepreneurship literature points out that a balance between exploration and exploitation leads to a good innovative performance of start-ups, yet how this balance can be achieved remains unknown. This study argues that the modularity principle, adopted from the complexity perspective, is the key to successful innovative strategies. By applying dynamic word embedding techniques to millions of documents from 119 business newspapers, magazines and patents, we construct a dynamic landscape of business discourse across 45 years. By locating new venture descriptions in this dynamic business landscape, we can observe how business elements are recombined within a given new venture and how innovative it was in its own time and historical context. Two different categories of recombination exist for each company: the first-order recombination that combines technical elements to approximate application elements to form basic functional blocks, and second-order recombination that recombines these blocks to satisfy more complex demands. Based on these measurements, we model the start-ups' birth, growth, and death with event history analysis. Our analysis reveals that radical second-order recombination is key to entrepreneurial success in the U.S.: the first-order process calls for exploitation while the second- order process requires exploration. Both exploitation and exploration matter for the new ventures, yet exploitation and exploration co-exist in innovative strategies through a specific architecture.