设椭圆方程为$\frac{x^2}{25}+\frac{y^2}{16}=1$，$F_1,F_2$分别是左右焦点，设椭圆上的动直线$A,B$过左焦点$F_1$.

 

(1)求$\triangle ABF_2$内心的轨迹；

 

(2)当$\triangle ABF_2$内切圆面积最大时，求直线$AB$的方程.

(1)根据题意有$F_1(-3,0),F_2(3,0)$，不妨设$A(x_1,y_1),B(x_2,y_2)$，且

$$\left\{ \begin{array}{l}{x_1} = 5\cos \alpha \\{y_1} = 4\sin \alpha\end{array} \right.\text{及}\left\{ \begin{array}{l}{x_2} = 5\cos \beta \\{y_2} = 4\sin \beta\end{array} \right.$$

由$A,B,F_1$三点共线可知

$$\frac{{4\sin \beta  - 4\sin \alpha }}{{5\cos \beta  - 5\cos \alpha }} = \frac{{4\sin \beta }}{{5\cos \beta  + 3}} \Leftrightarrow 3\left( {\sin \beta  - \sin \alpha } \right) = 5\sin \left( {\alpha  - \beta } \right).$$

即

$$5\cos \frac{{\alpha  - \beta }}{2} =  - 3\cos \frac{{\alpha  + \beta }}{2}\Leftrightarrow \tan \frac{\alpha }{2}\tan \frac{\beta }{2} =  - 4.$$

由椭圆第二定义可知

$$\begin{align*}\left| {A{F_2}} \right| &= 5 - 3\cos \alpha \\\left| {B{F_2}} \right| &= 5 - 3\cos \beta \\\left| {AB} \right| &= \left| {A{F_1}} \right| + \left| {A{F_1}} \right| = 3\left( {\cos \beta  + \cos \alpha }\right) + 10.\end{align*}$$

 

记$\triangle ABF_2$内心为$(x,y)$，则

$$\left\{ \begin{array}{l}x = \frac{{\left| {B{F_2}} \right|{x_1} + \left| {A{F_2}} \right|{x_2} + 3\left| {AB} \right|}}{{20}}\\y = \frac{{\left| {B{F_2}} \right|{y_1} + \left| {A{F_2}} \right|{y_2}}}{{20}}\end{array} \right.$$

化简整理得

$$\left\{ \begin{array}{l}x = \frac{{34\left( {\cos \beta  + \cos \alpha } \right) - 30\cos \alpha \cos \beta  + 30}}{{20}}\\y = \frac{{5\left( {\sin \beta  + \sin \alpha } \right) - 3\sin \left( {\alpha  + \beta } \right)}}{5}\end{array} \right.$$

注意到

$$\begin{align*}\sin \beta  + \sin \alpha  &= 2\sin \frac{{\alpha  + \beta }}{2}\cos \frac{{\alpha  - \beta }}{2} =  - \frac{6}{5}\sin \frac{{\alpha  + \beta }}{2}\cos \frac{{\alpha  + \beta }}{2}\\&=  - \frac{3}{5}\sin \left( {\alpha  + \beta } \right)\\\cos \beta  + \cos \alpha  &= 2\cos \frac{{\alpha  + \beta }}{2}\cos \frac{{\alpha  - \beta }}{2} =  - \frac{6}{5}{\cos ^2}\frac{{\alpha  + \beta }}{2}\\&=  - \frac{3}{5}\cos \left( {\alpha  + \beta } \right) - \frac{3}{5}\\\cos \alpha \cos \beta  &= \frac{1}{2}\cos \left( {\alpha  + \beta } \right) + \frac{1}{2}\cos \left( {\alpha  - \beta } \right)\\&= \frac{1}{2}\cos \left( {\alpha  + \beta } \right) + \frac{1}{2}\left( {2{{\cos }^2}\frac{{\alpha  - \beta }}{2} - 1} \right)\\&= \frac{1}{2}\cos \left( {\alpha  + \beta } \right) - \frac{1}{2} + \frac{9}{{25}}{\cos ^2}\frac{{\alpha  + \beta }}{2}\\&= \frac{1}{2}\cos \left( {\alpha  + \beta } \right) - \frac{1}{2} + \frac{9}{{25}}\frac{{\cos \left( {\alpha  + \beta } \right) + 1}}{2}\\&= \frac{{17}}{{25}}\cos \left( {\alpha  + \beta } \right) - \frac{8}{{25}}.\end{align*}$$

故可以再次化简成

$$\left\{ \begin{array}{l}x =  - \frac{{51}}{{25}}\cos \left( {\alpha  + \beta } \right) + \frac{{24}}{{25}}\\y =  - \frac{6}{5}\sin \left( {\alpha  + \beta } \right)\end{array} \right. \Rightarrow {\left( {\frac{{25x - 24}}{{51}}} \right)^2} + \frac{{25}}{{36}}{y^2} = 1\left( {x = \frac{{27}}{{25}},y = 0} \right).$$

(2)由题意

$$S = \frac{1}{2} \times 20r = \frac{1}{2} \times \left| {{F_1}{F_2}} \right| \times 4\left| {\sin \beta  - \sin \alpha } \right| = \frac{1}{2} \times 24\left| {\sin \beta  - \sin \alpha } \right|.$$

$$\begin{align*}\Rightarrow r &= \frac{6}{5}\left| {\sin \beta  - \sin \alpha } \right| = 2\left| {\sin \left( {\alpha  - \beta } \right)} \right|\\&= 4\left| {\frac{{\sin \frac{{\alpha  - \beta }}{2}\cos \frac{{\alpha  - \beta }}{2}}}{{{{\sin }^2}\frac{{\alpha  -\beta }}{2} + {{\cos }^2}\frac{{\alpha  - \beta }}{2}}}} \right|\\&= 4\left| {\frac{{\tan \frac{{\alpha  - \beta }}{2}}}{{{{\tan }^2}\frac{{\alpha  - \beta }}{2} + 1}}} \right|.\end{align*}$$

令$t = \tan \frac{{\alpha  - \beta }}{2}$，则$\left| t \right| = \left| {\tan \frac{{\alpha  - \beta }}{2}} \right| = \left| {\frac{{\tan \frac{\alpha }{2} - \tan \frac{\beta }{2}}}{{1 + \tan \frac{\alpha }{2}\tan \frac{\beta }{2}}}} \right| = \frac{{\left| {\tan \frac{\alpha }{2}} \right| + \frac{4}{{\left| {\tan \frac{\alpha }{2}} \right|}}}}{3} \ge \frac{4}{3}.$由此

$$r = \frac{4}{{\left| {t + \frac{1}{t}} \right|}} = \frac{4}{{\left| t \right| + \frac{1}{{\left| t \right|}}}} \le \frac{4}{{\frac{4}{3} + \frac{3}{4}}} = \frac{{48}}{{25}}.$$

当且仅当$\left| {\tan \frac{\alpha }{2}} \right| = \left| {\tan \frac{\beta }{2}} \right| = 2$时取等成立，这时有

$$\left\{ \begin{array}{l}{x_1} = 5\frac{{1 - {{\tan }^2}\frac{\alpha }{2}}}{{{{\tan }^2}\frac{\alpha }{2} + 1}} =  - 3\\{y_1} = 4\frac{{2\tan \frac{\alpha }{2}}}{{{{\tan }^2}\frac{\alpha }{2} + 1}} = \frac{{16}}{5}\end{array} \right.,\left\{ \begin{array}{l}{x_2} =  - 3\\{y_2} =  - \frac{{16}}{5}\end{array} \right.\text{或}\left\{ \begin{array}{l}{x_1} = 5\frac{{1 - {{\tan }^2}\frac{\alpha }{2}}}{{{{\tan }^2}\frac{\alpha }{2} + 1}} =  - 3\\{y_1} = 4\frac{{2\tan \frac{\alpha }{2}}}{{{{\tan }^2}\frac{\alpha }{2} + 1}} =  - \frac{{16}}{5}\end{array} \right.,\left\{ \begin{array}{l}{x_2} =  - 3\\{y_2} = \frac{{16}}{5}\end{array} \right.$$

此时直线$AB$的方程为$x=-3.$

 

另附西哥的解答：

解:引理:设$\Delta ABC$的三边为$a,b,c$,且顶点$A(x_{1},y_{1}),B(x_{2},y_{2}),C(x_{3},y_{3})$, 内心坐标$I(x,y)$,则有 $$\begin{cases}x=\dfrac{ax_{1}+bx_{2}+cx_{3}}{a+b+c}\\y=\dfrac{ay_{1}+by_{2}+cy_{3}}{a+b+c}\end{cases}$$

设$A(x_{1},y_{1}),B(x_{2},y_{2}),I(x,y)$,则$AB$方程为$y=k(x+c)$与椭圆联立得 $$(b^2+a^2k^2)x^2+2a^2ck^2x+a^2c^2k^2-a^2b^2=0$$

  则 $$x_{1}+x_{2}=-\dfrac{2a^2ck^2}{b^2+a^2k^2},x_{1}x_{2}=\dfrac{a^2c^2k^2-a^2b^2}{b^2+a^2k^2}$$

所以 $$y_{1}+y_{2}=k(x_{1}+x_{2}+2c)=\dfrac{2b^2ck}{b^2+a^2k^2}$$ $$x_{1}y_{2}+x_{2}y_{1}=2kx_{1}x_{2}+kc(x_{1}+x_{2})=-\dfrac{2a^2b^2k}{b^2+a^2k^2}$$

  因为 $$|AF_{1}|=a+ex_{1},|AF_{2}|=a-ex_{1},|BF_{1}|=a+ex_{2},|BF_{2}|=a-ex_{2}$$

所以 $$|AB|=|AF_{1}|+|BF_{1}|=2a+e(x_{1}+x_{2})$$

所以 $$\begin{align*}&x=\dfrac{(a-ex_{2})x_{1}+(a-ex_{1})x_{2}+[2a+e(x_{1}+x_{2})]c}{4a}\\&=\dfrac{(a+ec)(x_{1}+x_{2})-2ex_{1}x_{2}+2ac}{4a}\\&=\dfrac{\dfrac{a^2+c^2}{a}\left(-\dfrac{2a^2ck^2}{b^2+a^2k^2}\right)-\dfrac{2c}{a}\cdot\dfrac{a^2c^2k^2-a^2b^2}{b^2+a^2k^2}+2ac}{4a(b^2+a^2k^2)}\\&=\dfrac{4ab^2c-4ac^3k^2}{4a(b^2+a^2k^2)}\\&=\dfrac{b^2c-c^3k^2}{b^2+a^2k^2}\end{align*}$$

所以 $$k^2=\dfrac{b^2c-b^2x}{a^2x+c^3}$$

$$\begin{align*}y&=\dfrac{(a-ex_{2})y_{1}+(a-ex_{1})y_{2}}{4a}=\dfrac{a(y_{1}+y_{2})-e(x_{1}y_{2}+x_{2}y_{1})}{4a}\\&=\dfrac{a\cdot\dfrac{2b^2ck}{b^2+a^2k^2}-\dfrac{c}{a}\left(-\dfrac{2a^2b^2k}{b^2+a^2k^2}\right)}{4a}\\&=\dfrac{b^2ck}{b^2+a^2k^2}\end{align*}$$

所以

$$\begin{align*}y^2&=\dfrac{b^4c^2k^2}{(b^2+a^2k^2)^2}=\dfrac{b^2c^2\cdot\dfrac{b^2c-b^2x}{a^2x+c^3}}{\left(b^2+a^2\cdot\dfrac{b^2c-b^2x}{a^2x+c^3}\right)^2}=\dfrac{\dfrac{b^6c^2(c-x)}{a^2x+c^3}}{\left[\dfrac{(a^2x+c^3)b^2+a^2(b^2c-b^2x)}{a^2x+c^3}\right]^2}\\&=\dfrac{\dfrac{b^6c^2(c-x)}{a^2x+c^3}}{\left[\dfrac{b^2c(a^2+c^2)}{a^2x+c^3}\right]^2}=\dfrac{b^2(c-x)(a^2x+c^3)}{(a^2+c^2)^2}\end{align*}$$

所以

$$(a^2+c^2)^2y^2=(b^2c-b^2x)(a^2x+c^3)=b^2(cb^2x+c^4-a^2x)$$

即

$$a^2b^2x^2+(a^2+c^2)^2y^2-cb^4x-b^2c^4=0$$

此问题中,令$a=5,b=4$,则内心I的轨迹方程为 $$100x^2+289y^2-192x=324$$

(2)设直线$AB$方程为$x=ky-3$与椭圆联立得 $$(16k^2+25)y^2-96ky-256=0$$

所以 $$y_{1}+y_{2}=\dfrac{96k}{16k^2+25},y_{1}y_{2}=-\dfrac{256}{16k^2+25}$$

所以 $$S_{\Delta ABC}=\dfrac{1}{2}|F_{1}F_{2}||y_{1}-y_{2}|=3\sqrt{(y_{1}+y_{2})^2-4y_{1}y_{2}}=3\sqrt{\dfrac{96^2k^2}{(16k^2+25)^2}+\dfrac{4\times}{16k^2+25}}$$ $$S_{\Delta ABC}=960\sqrt{\dfrac{k^2+1}{(16k^2+25)^2}}$$

令$k^2+1=t\ge1$,则有 $$\dfrac{k^2+1}{(16k^2+25)^2}=\dfrac{t}{(16t+9)^2}=\dfrac{t}{256t^2+288t+81}=\dfrac{1}{256t+\dfrac{81}{t}+288}\le\dfrac{1}{625}=\dfrac{1}{25^2}$$

即$S_{\Delta ABC}\le 38.4$当且仅当$k=0$时取等.故此时$AB$方程为$x=-3$.

$$\begin{align}A &= \int_{{y_1}}^{{y_2}} {\left( {my + l - \sqrt {{y^2} + 2} } \right)dy}  = \left[ {\frac{{m{y^2}}}{2} + ly - \left( {\frac{{y\sqrt {{y^2} + 2} }}{2} + \ln \left( {y + \sqrt {{y^2} + 2} } \right)} \right)} \right]_{{y_1}}^{{y_2}}\\&= \frac{m}{2}\left( {y_2^2 - y_1^2} \right) + l\left( {{y_2} - {y_1}} \right) - \left( {\frac{{{y_2}\sqrt {y_2^2 + 2} }}{2} - \frac{{{y_1}\sqrt {y_1^2 + 2} }}{2}} \right) - \ln \frac{{{y_2} + \sqrt {y_2^2 + 2} }}{{{y_1} + \sqrt {y_1^2 + 2} }}\\&= \frac{m}{2}\left( {y_2^2 - y_1^2} \right) + l\left( {{y_2} - {y_1}} \right) - \left( {\frac{{{x_2}{y_2}}}{2} - \frac{{{x_1}{y_1}}}{2}} \right) - \ln \frac{{{x_2} + {y_2}}}{{{x_1} + {y_1}}}\\&= \frac{m}{2}\left( {y_2^2 - y_1^2} \right) + l\left( {{y_2} - {y_1}} \right) - \left[ {\frac{m}{2}\left( {y_2^2 - y_1^2} \right) + \frac{l}{2}\left( {{y_2} - {y_1}} \right)} \right] - \ln \frac{{{x_1}{x_2} - {y_1}{y_2} + {x_1}{y_2} - {x_2}{y_1}}}{2}\\&= \frac{l}{2}\left( {{y_2} - {y_1}} \right) - \ln \frac{{\left( {{m^2} - 1} \right){y_1}{y_2} + ml\left( {{y_1} + {y_2}} \right) + l\left( {{y_2} - {y_1}} \right) + {l^2}}}{2}\end{align}$$

又