A standard approach to the analysis of free trade agreements (FTAs) involves simulating a general equilibrium model of the global economy in response to shocks that reflect trade cost liberalisation. A few methods are common to estimate the scale of these essential trade shocks, but little formal research has gone into evaluating the success of these different approaches. Developing a systematic approach to the evaluation of FTA trade shock estimates is essential to informed policy making in international trade, as well as to the academic literature on the evaluation of FTAs. This paper contributes to the existing small literature focused on estimating trade impacts for use in economic modelling of trade. It extends a framework first used in a paper by [Baier et al., 2019]. This method presented a novel way to evaluate ex-ante trade shock estimation methodologies by posing the issue as a prediction problem to predict ex-post FTA shocks identified through a first stage gravity model. In this paper I extend the original paper’s ex-ante analysis by making use of a wider dataset, as well as adopting recent methodological innovations from the machine learning literature. I find that amongst the most important considerations in making ex-ante FTA impact estimates are whether a model is trained on data excluding outliers and the type of machine learning model chosen. Furthermore, the broader set of data used is found to be important in improving success.