A viral Instagram account that includes a blonde Army service member named Jessica Foster, posing alongside world leaders, racked up greater than 1 million followers earlier than it was revealed to be pretend.
This is just one instance in a rising wave of AI-generated personas utilizing navy identification to construct audiences and generate revenue on-line.
The directors behind Army Phony, a watchdog group that tracks fraudulent navy claims, described “digital stolen valor” as the web equal of carrying medals you didn’t earn, utilizing exaggerated or fabricated credentials to achieve respect, sympathy or alternative that may in any other case belong to another person.
They draw a distinction between violations of the federal Stolen Valor Act, which contain falsely claiming sure navy honors, such because the Purple Coronary heart or Silver Star, for tangible profit and broader types of impersonation that won’t meet that authorized threshold however are nonetheless broadly known as “stolen valor.”
The rise of AI-generated influencers and impersonated service members is exposing what some observers are starting to see as a brand new type of “digital stolen valor,” the place artificial personas undertake the credibility of navy service or different trusted professions like nursing, to draw followers, drive engagement, and, in some circumstances, generate revenue.
Whereas impersonation and fraud on-line are nothing new, advances in synthetic intelligence are making these identities simpler to create, more durable to detect, and simpler at exploiting belief.
The ‘Emily Hart’ Account
One such account, working below the identify “Emily Hart,” constructed a big following by pairing political messaging with curated life-style content material, finally directing customers towards paid grownup content material subscriptions.
The persona was later revealed to be AI-generated, created by a 22-year-old medical pupil, based on a report by Wired.
Recognized by the pseudonym “Sam,” the creator advised the outlet he started experimenting with AI-generated pictures as a solution to earn additional revenue whereas at school and save towards a possible transfer to america after commencement.
Based on the report, he used Google’s Gemini AI to refine the idea, finally creating a fictional persona tailor-made to a conservative-leaning viewers. The chatbot prompt that such audiences, significantly older males within the U.S., are usually extra financially engaged and dependable, influencing the course of the account.
The Division of Protection declined to remark immediately on the pattern, however referred inquiries to federal regulation enforcement.
“As impersonating a member of the armed forces is a violation of federal regulation, we refer you to the FBI,” a Pentagon official advised Army.com. As of this publication, the FBI has not responded to requests for remark.
Authorized Precedent
Authorized specialists say the excellence between protected speech and punishable conduct typically comes right down to intent and revenue. Merely claiming to be a service member on-line, even falsely, can fall below constitutionally protected speech, based on Eugene Volokh, a senior fellow on the Hoover Establishment and professor of regulation emeritus at UCLA.
However that safety has limits, the professor defined to Army.com, citing the authorized case U.S. v. Alvarez.
“Merely claiming to be a service member, with none business dimension, and easily in search of fame or affect, is mostly constitutionally protected,” Volokh stated.
“The place false claims are made to impact a fraud or safe moneys or different worthwhile issues … it’s properly established that the federal government might limit speech with out affronting the First Modification,” Volokh advised Army.com, citing the Supreme Courtroom’s resolution in United States v. Alvarez, a 2007 case involving a person named Xavier Alvarez who advised a crowd that he was a 25-year Marine veteran and was awarded the Congressional Medal of Honor—all fabricated data.
In different phrases, whereas an AI-generated persona posing as a service member to achieve consideration or affect could also be protected, utilizing that very same identification to solicit cash by way of subscriptions, donations or merchandise may expose the operator to civil and even felony legal responsibility.
That distinction applies no matter how the persona is created, which means false claims embedded inside AI-generated accounts are handled no otherwise below the regulation than these made by actual people.
Based on Volokh, Alvarez acknowledged that “the place false claims are made to impact a fraud or safe moneys or different worthwhile issues, say, provides of employment, it’s properly established that the federal government might limit speech with out affronting the First Modification.”
“Thus, attempting to get cash or different valuables by way of realizing and materials falsehoods, together with by claiming to be a member of the navy, is punishable,” he added. “It may result in lawsuits, civil enforcement and even felony legal responsibility.”
Platforms Battle to Hold Tempo
Regardless of platform guidelines requiring disclosure of AI-generated content material, enforcement stays inconsistent. Most of the accounts driving engagement seem unlabeled or are eliminated solely after gaining important traction, permitting them to construct giant audiences and, in some circumstances, generate income earlier than being taken down.
Meta, which owns Instagram, has insurance policies requiring customers to reveal AI-generated or manipulated content material, however the firm has not publicly detailed how these guidelines are enforced at scale or how rapidly probably misleading accounts are recognized.
Meta didn’t response to inquiries from Army.com.
For watchdog teams, the priority is not only that these accounts exist, however that they’re changing into more durable to establish.
Directors behind Army Phony famous that AI-generated pictures can obscure or distort key particulars, akin to rank insignia or uniform accuracy, that skilled observers typically depend on to identify fraudulent claims.
The accounts themselves are sometimes designed to sign authenticity as rapidly as attainable, pairing visible cues like uniforms or skilled settings with messaging tailor-made to particular audiences. In a number of latest circumstances, AI-generated personas adopted politically aligned identities alongside navy or healthcare roles—a mix that may speed up engagement by reinforcing familiarity and belief.
That dynamic might assist clarify why some accounts proceed to draw followers even when questions on authenticity emerge. The enchantment just isn’t at all times rooted in whether or not the persona is actual, however whether or not it displays beliefs, identification or values that resonate with an viewers.
