October 11, 2018: The U.S. Army wants to provide troops in combat zones with social media analysis tools that would enable them to automatically gather information from local Internet social media to see what is happening in the area they are operating. This would be near real-time analysis with results available in English. This sort of thing is now possible because of advances in translation and social media analysis software. Intelligence organizations and commercial firms have been developing this sort of capability for over a decade. Currently, military use of these professional tools is restricted and their analyses are usually classified (or considered business secrets). But that has been changing. For example, the Israeli domestic intel agency (Shin Bet) revealed that so far in 2018 its intel monitoring efforts on social media had played a major role in preventing over 250 terrorist attacks. Many big-city police departments use commercially available analysis software to monitor local social media users for useful data on past or potential crimes. Before that police would simply search the social media for telltale signs of crimes committed or about to be. In Ukraine and throughout the Middle East local security forces increasingly use this capability either manually or with specialized software. Now, it appears, it is time to provide the troops with similar tools. Apparently, there has already been some informal and unofficial action in this area by individual army units in combat zones. U.S. SOCOM (Special Operations Command) has been using such tools for years.

Early on social networking sites and Internet activity analysis were recognized as a major tool for intelligence agencies. There is an irony in this because sites like Facebook and Twitter are also hailed as catalysts for revolution and social change. While that's true, these sites have also been a big help to intelligence and police organizations. This can have fatal consequences in dictatorships, where the police and intel groups can use data gathering and analysis tools (developed for marketing via the Internet) to find people who are protesting or rebelling against the government. Even if these Facebook users are using codes and pseudonyms to remain hidden, the scanning and analysis tools can often uncover them. Twitter traffic can also be analyzed for useful information on who is doing what and where they are.

Social networking sites are thus a double-edged sword. They can be used to organize, inform, and mobilize large groups. But in doing this you provide the secret police a lot of information you would rather not share with them. Islamic terror groups advise their members to avoid social networking sites, but that has proved impossible to enforce. Social networking was designed to be alluring, as well as useful, especially to the young and niche groups. For young revolutionaries, this can be a fatal attraction. Social networking sites can be used to organize, inform, and mobilize large groups. But in doing this you provide the secret police a lot of information you would rather not share with them.

Intelligence agencies, especially in the United States, were quick to adopt commercial techniques used for BI (Business Intelligence, or corporate espionage) and data mining operations and applying it to the massive quantities of real-time data on the Internet. The CIA developed software to gather all this Internet information, filter and organize it and then turn it over to analysts to be sorted out, or, in many cases, translated more accurately. That last bit was necessary because machine translation software can automatically translate all those tweets and postings so that stuff can be identified and put in a database. But in order to get really useful (to the CIA) intelligence, you need skilled linguists and analysts to double check and also find out if the selecting and sorting software needs to be tweaked (it often does).

This massive, real-time combing of social media and open (to anyone) message traffic has yielded a much more accurate and timely analysis of political, religious, cultural, and military trends worldwide. It has also made the deployment of agents and other scarce resources (reconnaissance and electronic eavesdropping satellites, aircraft, and ships) more effective.

In the United States the FBI, Homeland Security, and military intelligence have similar data gathering and analysis systems for gathering all sorts of useful information. Other nations are establishing similar systems, often using commercial software sold to marketing firms and large corporations.

These capabilities increasingly became newsworthy. In 2016 the American government took credit for the recent decline in ISIL (Islamic State in Iraq and the Levant) activity on social media. On Twitter, for example, messages supporting or mentioning ISIL in 2016 were about half as numerous now as they were in 2014. By 2016 there was a lot more anti-ISIL commentary and traffic among Arab speaking Internet users. While some of this decline in pro-ISIL activity is due to public opinion turning against ISIL because of their continued attacks on Moslem women and children and any Moslem who does not agree with them there were other trends developing as well. For example, Western governments have learned (often by trial and error) how to use the law, popular attitudes among Moslems towards Islamic terrorism and some commonly used Internet marketing and publicity techniques against Islamic terror groups. All these efforts have been underway for years and have reported much progress since 2015.

Before 2015 a lot the basic work was done by SOCOM which pioneered the use on commercial technology to gain an edge as Islamic terrorists have moved a lot of the combat to the Internet. Islamic terrorists used the Internet for a lot of their recruiting, fundraising, training and carrying out attacks or discussing tactics in general. SOCOM used commercial Internet marketing software systems to better (and more quickly) analyze what Islamic terrorists were doing on the Internet. This was nothing new for SOCOM, which has been informally using social networking sites and Internet activity in general to find, monitor and sometimes manipulate terrorist suspects. This has been going on since the late 1990s but as time went on SOCOM found that formal and analytic techniques produced better results.

The U.S. Army Special Forces has one of the largest collections of experienced counter-terrorism operators who know the culture and languages of areas where there is a lot of Islamic terrorist activity. These soldiers have spent years learning about cultures and languages and honed that knowledge by actually operating in those areas. This is now being used to develop rapid analysis of Internet data (discussion topics, direction of discussions, traffic analysis) and quickly determine how best to respond. Many people do not realize that SOCOM had always had lots of psychological war specialists and these specialists set up message centers staffed with locals who can start posting new types of messages in response to detected trends. What SOCOM was seeking were ways to derail Islamic terrorists propaganda (especially the ones involving mass murder or beheadings of civilians) and capitalizing on it to mobilize local (where the Islamic terrorists operate) public opinion against these atrocities. This played a major role in developing anti-ISIL Internet messages that resonated with Moslem audiences ISIL was depending on.

By playing up the extent of harm ISIL was doing to Moslem children and Moslem culture, in general, the West finally had an anti-ISIL message that could not be dismissed as “Western propaganda.” That was largely because of the use of people literate in Arabic and other languages popular with many Islamic terrorists in Africa, Afghanistan, Pakistan and Southeast Asia to get on pro-terrorist message boards and post messages designed to weaken the ISIL propaganda.

Finally, the government has taken better advantage of laws in many countries that forbid the kind of “kill, kill, kill” message ISIL regularly uses. ISIL turned out to be very vulnerable here because there was ample evidence, often posted to the web by ISIL, that the vile ISIL message was not idle chatter but deadly serious and therefore illegal and subject to bans on ISIL using social media in many countries.