Artificial Intelligence (AI) is the new buzz word in the research world, and rightly so. There are many questions surrounding the use of AI in research. Can AI auto-generate quality profiles? Are sources used by AI to generate text reliable and accurate? How accurate are AI generated wealth ratings?
As a researcher, I was naturally curious about AI, how it works, and if it could be helpful in my research. So, I tested two free online tools, Perplexity, used for quickly finding information on a topic; and AI Checker, a tool for detecting AI generated text. I used a real-life situation I came across in my research.
The Test
I recently researched a prospect who owns a California company that was established in the early 1900s. The company’s name indicated it could possibly be, or formally was, a copper mine. I know next to nothing about copper mining in California, but I had to find out if this company was in fact an active mine. The challenge was to quickly gather then read about and understand the industry so I could write an intelligent paragraph on copper mining and its wealth implications.
I found good information on the history of copper mining in California using Google. However, it took some time to sort through results, read the articles, synthesize the information, then compose an informative summary for the profile. When research requests are timely (as they often are!) this approach may not be the most efficient way to gather information.
Experiment
For my experiment, I asked Perplexity about “copper mining in California.” In less than a second, Perplexity provided me with exactly what I was looking for – an overview of copper mining in California. All and all, I was pleased with the results. Below is the text Perplexity returned.
Perplexity AI Generated Text on Copper Mining in California.
Source: https://www.perplexity.ai/search/d0aec541-9ea2-4f88-9a7c-ae54b2372e63?s=u
Review
The content, for the most part, was what I found in my original research. However, Perplexity’s results came back much, much quicker! The writing style, although not bad, was a bit stilted in some sections. It is definitely not how I would have written the text.
One fact Perplexity stated was incorrect. According to Perplexity, “California is the largest producer of copper in the western belt.” That may have been true at one time. However, according to usgs.gov, “Arizona leads in copper production.”
Notice the superscript numbers after some of the text? Those numbers referred to the sources Perplexity used to generate the text. A few sources were the ones I found in my original research. Even better, Perplexity provided links to the sources, so I could verify the information or read it in greater detail, if desired. This is a handy feature!
Sources used by Perplexity for this experiment:
Next, I wanted to verify the sources. Were the sites real and reputable? Was the information reliable? Was the information aggregated from different sources? Let’s examine them.
- The Stanford Libraries site is an education site. The California Water Boards is a government site. Sites such as these do tend to be reputable and reliable.
- The Western Mining History site appears to be managed by an individual or a small group. Nevade Outback Gems is an online gem store that happens to provide information on mining as one of its many links on its site. Sites such as these may or may not be consistently review the supplied information for accuracy or legitimacy.
- Wikipedia is “an online open-content collaborative encyclopedia” that the general public can add articles and edit at will. Because of this, the site acknowledges that it “cannot guarantee the validity of the information found here.” (source: Wikipedia General Disclaimer). Just something to keep in mind when you search Wikipedia.
I took this experiment one step further to see if the text would stand up under scrutiny. I ran Perplexity’s text through AI Checker. The tool indicated that “Your Text is Most Likely AI/GPT generated, may include parts written by Human.” It felt that 43.62% of the text was AI/GPT generated. The tool even highlighted the sections it suspected of being generated by AI.
Just for fun, I ran the identical Perplexity text again two more times using AI Checker. This time AI Checker felt that only 37.25% of the text was AI/GPT generated. The third time it was 41.39%. Taking the average of the three results, AI Checker felt that only about 40% of the text was AI generated. The range between the highest and lowest percentage was 6.37. Not an extremely wide range, but still noteworthy as the text was the same.
The AI Checker results are interesting in two ways. First, even though the text was 100% AI generated, each time AI Checker didn’t recognize it as such. Second, the exact same text produced three different results! This suggests that the AI Checker tool may not always be reliable when checking for AI generated text.
Application
You may ask what this information has to do with a research profile. Well, I learned that a) this company was established in the early 1900s, which coincided with the height of copper mining in California; b) the company is located in a county that was a known area for copper deposits; c) based on that knowledge, it can be assumed this was an active mine at one point; and d) as copper mining is no longer an active industry in California, it is likely that the still-active company has moved on to other business ventures — a family holding company, as I later discovered.
With my newfound knowledge of copper mines, I was able to provide a nice summary paragraph to inform the client of this aspect of the prospect’s business interests and an indication of their multi-generational wealth.
Conclusion
- Perplexity provided a nice overview of the chosen topic. It quickly found enough material for me to write intelligently on the subject matter.
- It is best to take the information with a grain of salt and verify the sources used, especially when the sources, such as Wikipedia, can be edited by anyone.
- Definitely do not use the information verbatim in a profile. First, the writing may be stilted. Second, there is always the possibility the reader may decide to run the text through an AI checker tool. You would not want to be caught using AI generated text in your profile. How embarrassing would that be!
As a research tool, Perplexity is great to quickly gather and summarize information on a particular topic, thus saving the researcher valuable time. The cited sources provide an opportunity to ‘dig deeper’ (pun intended) into the chosen topic. I would not hesitate to use Perplexity again in a similar situation.
This is only one scenario. I wonder how Perplexity will respond to requests such as compiling a prospect’s work biography or providing a summary of a prospect’s charitable giving. I also wonder if another AI generative software, such as ChatGPT, would return similar or different results. The experiment continues….