According to Josephus, Herod had started the rebuilding of the Temple in the eighteenth year of his reign, which was about 20-19 B.C.  Work continued on it until its completion in 64 A.D.  The Jews here state that the work has been proceeding for 46 years, which would indicate that the year was 27 or 28 A.D.

The Jews are very obviously sneering at the abilities of this peasant (not a member of the Sanhedrin nor even an architect) to rebuild a destroyed Temple at all, much less in 3 days.  The irony was that the task they thought to be impossible was much easier than the one to which He was referring. They misunderstood His words and His power. Actually, Messiah was prophesied to build the Temple (Zech 6:12) which Christ did by building the church (Eph 2:22, 1Peter 2:5, 1Cor 3:16). As to his power, He built everything that exists in 6 days, how big a deal would it be for him to rebuild the Temple in 3 days?

Zech 6:12  Tell him this is what the LORD Almighty says: ‘Here is the man whose name is the Branch, and he will branch out from his place and build the temple of the LORD.

Eph 2:22  And in him you too are being built together to become a dwelling in which God lives by his Spirit.

1 Peter 2:5  you also, like living stones, are being built into a spiritual house to be a holy priesthood, offering spiritual sacrifices acceptable to God through Jesus Christ.

1 Cor 3:16  Don’t you know that you yourselves are God’s temple and that God’s Spirit lives in you?