I can say unequivocally GO FOR IT! I had three years of post-doc experience in the US and took a post-doc in Europe partly as a last resort following an unsuccessful TT job search in the US. As in your case, this European post-doc was in a new area that was a specialty for the department where I worked, and an area that is difficult to get funding for in the US.
I also did not speak the native language but found this to be no problem—in my experience, almost everyone under 50 in Western Europe speaks English well, and everyone in academia speaks English perfectly (though they are often too modest to admit it!). Of course, a full experience of the country requires some language ability, but in my case the university paid for me to take language classes, which was a big help. I would ask about this possibility.
I had been very concerned that US search committees would not be willing to foot the bill to fly me over to interview for faculty positions. But in the most recent hiring cycle, I got two on-campus interview offers for TT jobs. I was offered and took one of those positions and am now back in the US. This leads me to believe that working abroad can give you an edge for jobs—despite the added recruitment costs. You may appear especially open-minded or exotic to a search committee, especially if you are working in a world nexus for your emerging area of specialization.
Again speaking in generalizations, in Europe there is a lot of freedom to pursue research whereas teaching is often deemphasized or even something of an afterthought, even compared to US R1 schools. Depending on your plans for the future, this could be good or bad. In any case, it would be good to ask about the teaching expectations beforehand since you may have a significant teaching load sprung upon you (as a post-doc) just before the semester starts!
It is also the case that many European countries have very generous social programs, which often apply to foreigners as well. My partner had a child during our stay in Europe and for the subsequent year we received over 1200 Euro per month in cash support, not to mention excellent socialized health care. There are of course differences from country to country, and cost of living can also vary widely, but my sense is that conditions for families are much better than they are in the US (at least for postdocs). This would be another thing to ask about (depending on your situation).
If by chance your potential position is in Switzerland, you are in luck. Although this is not where I was working, I am told that academics there have the most plum positions, with the highest pay and best benefits. My understanding is that Germany, Austria, France, Holland and Belgium are also great for academics, along with certain institutions in Italy.
Moving to Europe is not without its challenges. The physical move itself and the separation from friends and family can be very difficult. But on the whole I think it is a great opportunity.