LT tires will generally be more durable, but much heavier than their P-metric equivalents. Your fuel mileage may suffer.
For some reason, I thought the Michelin Defender LTX in our stock 20" wheel size was an LT tire, but I see now that it's a P tire. They do make LT variants, but not in our stock size. No use in upsizing if you're not going off-road... especially if you're towing, in which case stock (or even smaller) size will put less strain on your drivetrain and sap less torque (bigger tires reduce torque, acceleration, braking power).
https://www.michelinman.com/tires/defender/defender-ltx-m-s.html
One downside of an All-Terrain tire (vs. All-season / H/T) is that there are bigger voids between the treads, so on-road non-towing dry traction will be somewhat compromised. This has never bothered me, even being a lead-footed / ham-****** driver. While towing my 6500 lb trailer, I've smashed the brakes a couple times (oops!) and have never engaged ABS, so tire-traction doesn't appear to be the weak link, even with All-terrain tires...
I personally really like the Cooper AT3 XLTs - they look nice, they're durable (LT rated, Load Range E), they're surprisingly quiet on-road, and they are capable enough off-road to keep my family and me from getting stuck in most situations - even some that are uncomfortable.
If brand value and your trust in the Michelin name is important to you, and you're not doing any off-roading, then the Defender LTX look like a good choice. It's just not for me.