The paper presents results of mathematical and numerical simulation of the stress-strain state of a dental system consisting of soft and bone tissue of a jaw, as well as dentition restored by the strut arc prosthesis. The main purpose of this study is to construct a methodology for determining the individual optimal forms of an element of the arc prosthesis, namely, of an underlayment. Here, the optimal form of the underlayment is understood as the rational distribution of its thickness, which allows to minimize the negative influence of the vertical load on the tissues of the prosthetic bed. The mathematical model of such problem of underlayment form optimization is suggested. This model belongs to a class of multicriteria problem of domain optimization. Here the criteria characterize stress values and uniformness of their distribution at a contact boundary “lining – soft tissues”. Some approaches to development of constructive methods of solving the problem are discussed. In particular, one numerical algorithm of approximate solution of this problem is suggested. The algorithm takes into account specific of the problem. The linear convex combination of the criteria and the special variations of a lining boundary are used. The results of numerical experiments with using real data are presented.