Yeah, so the first thing I should clarify here is that there is no technical difference between a "language" and a "dialect". Linguists often use the term "language variety" to be maximally clear on this point. French is a language variety, Bavarian (traditionally called a dialect of German) is a language variety, California English is a language variety. One language variety can be part of another: California English is a variety of English, "English" itself being a broader variety with many sub-varieties. We might further split California English, perhaps into Southern California English, Northern California English, and California Central Valley English, or some such. In the extreme, we can look only at the speech of an individual person; this is called an idiolect. Every human being who speaks a language speaks in their own unique idiolect, which differs from the speech of other humans in various ways. An idiolect is also a language variety.
Now, linguists do make a distinction, a very important distinction, between native and non-native speech. Roughly, a native speaker is someone who acquired a language by exposure during childhood. They were not explicitly taught the language, but picked it up by virtue of being surrounded by people who speak it. Human children seem to have various sorts of special cognitive mechanisms for acquiring language in this way, many of which we lose as we get older. This early period of life in which humans are primed for language acquisition is called the "critical period". There is a lot of debate about what exactly defines the critical period and when it ends (it's more of a gradual taper than a sharp cutoff), but there is basically no debate over the idea that children and adults have at least some fundamental differences when it comes to language learning. A non-native speaker, then, is someone who learned a given language in adulthood, after the critical period of language acquisition.
When linguists speak of a language variety, by fiat they take that variety to be defined by the speech of its native speakers. That is to say: the grammar of English is defined to be that set of rules which describes the speech of native English speakers. Where different varieties of English disagree, a thorough descriptive grammar will make note of that variation, and researchers will zoom in and study on its own terms the grammar of each relevant sub-variety. Every human is, by definition, a perfectly fluent speaker of their own idiolect in any language they acquired during childhood.
So, you asked about the difference between native and non-native "accents". In light of all the above, there are two differences:
- First, there is the difference between native and non-native speech in general. Non-native speech is characterized by certain artifacts of the adult language learning process, including carry-over from one's native language(s), which broadly do not affect native speech. Thus, a non-native "accent" is different from a native "accent" in various empirical ways which are pinned down and studied in the field of second language acquisition.
- By virtue of the way we have set our definitions up, above, native "accents" differ from non-native ones in that a native "accent" is in fact synonymous with a language variety; rather than being an imperfect specimen of some predefined standard language, it is a definitionally perfect exemplar of a particular linguistic system in its own right.
You could, of course, take up the linguistic system represented by the speech of some adult learner as an object of study in its own right, and some people do. But by and large, the standard which is taken up in linguistics is "language varieties are defined by the speech of their native speakers". I think this is a quite reasonable place to draw a line, especially in light of the empirical differences, as mentioned, between native and non-native speech.