And indeed they used to say;"If we had a reminder as had the men of old, we would have indeed been the chosen servants of Allah!"
means, `they used to wish -- before you, O Muhammad, came to them -- that they would have someone to remind them about Allah and what happened in earlier times, and to bring them the Book of Allah.'
This is like the Ayat;
وَأَقْسَمُواْ بِاللَّهِ جَهْدَ أَيْمَـنِهِمْ لَيِن جَأءَهُمْ نَذِيرٌ لَّيَكُونُنَّ أَهْدَى مِنْ إِحْدَى الاٍّمَمِ فَلَمَّا جَأءَهُمْ نَذِيرٌ مَّا زَادَهُمْ إِلاَّ نُفُوراً
And they swore by Allah their most binding oath that if a warner came to them, they would be more guided than any of the nations (before them); yet when a warner came to them, it increased in them naught but flight (from the truth). (35;42)
and,
أَن تَقُولُواْ إِنَّمَأ أُنزِلَ الْكِتَـبُ عَلَى طَأيِفَتَيْنِ مِن قَبْلِنَا وَإِن كُنَّا عَن دِرَاسَتِهِمْ لَغَـفِلِينَ
أَوْ تَقُولُواْ لَوْ أَنَّأ أُنزِلَ عَلَيْنَا الْكِتَـبُ لَكُنَّأ أَهْدَى مِنْهُمْ فَقَدْ جَأءَكُمْ بَيِّنَةٌ مِّن رَّبِّكُمْ وَهُدًى وَرَحْمَةٌ فَمَنْ أَظْلَمُ مِمَّن كَذَّبَ بِأيَـتِ اللَّهِ وَصَدَفَ عَنْهَا سَنَجْزِى الَّذِينَ يَصْدِفُونَ عَنْ ايَـتِنَا سُوءَ الْعَذَابِ بِمَا كَانُواْ يَصْدِفُونَ
Lest you should say;
"The Book was sent down only to two sects before us, and for our part, we were in fact unaware of what they studied."
Or lest you should say;
"If only the Book had been sent down to us, we would surely have been better guided than they."
So now has come unto you a clear proof from your Lord, and a form of guidance and a mercy. Who then does more wrong than one who rejects the Ayat of Allah and turns away therefrom We shall requite those who turn away from Our Ayat with an evil torment, because of their turning away. (6;156-157)
Allah says here;
فَكَفَرُوا بِهِ فَسَوْفَ يَعْلَمُونَ