MM 2017: Member Proposal to Assist with the Creation of an Alternative to Corporate Controlled Internet Search

Membership Meeting
1

Proposal from member Angie D’Urso:

Question:

“What does our movement do to create an alternative to the corporate-controlled Internet?”

Answer:

– Educate its members and all others about corporate Google’s political control of humanity’s access to all information through its Internet search engine, and join others already working towards creating an alternative, distributed, neutral search engine for humanity’s access to all information through the Internet. In short, ASSIST WITH THE CREATION OF AN ALTERNATIVE TO CORPORATE CONTROLLED INTERNET SEARCH

Some Quick Facts May Firsters Need About Google
which I’ll address in this document here with examples and references:

Corporate Google has a monopoly on search - and alternative search engine websites use Google’s search engine results - so simply using other internet search engines is NOT a solution to the problem addressed here & all other search engines are following Google’s lead. First, Google DEGRADED its search engine so that even when you use quotation marks to find specific content you knew was out there, Google ignores them and no longer offers up what it previously did – instead it offers up what it thinks you really want, or what it wants you to want, not what your search query tells Google you want. If you can’t find it on Google, does it exist? (For background, see my article
www.tinyurl.com/Searching-Internet-Google-Evil )

Now, Google has moved on to blatantly altering its secret search algorithms to hide left political content on the net. EXAMPLES:

“Beginning in April of this year, Google began manipulating search results to channel users away from socialist, left-wing, and anti-war publications, and directing them instead towards mainstream publications that directly express the views of the government and the corporate and media establishment (i.e., the New York Times, Washington Post, etc.)”
Copied from: An open letter to Google: Stop the censorship of the Internet! Stop the political blacklisting of the World Socialist Web Site! - World Socialist Web Site - http://www.wsws.org/en/articles/2017/08/25/pers-a25.html
(emphasis supplied).

See also this from Salon:

“Liberal and progressive sites appear to be among the victims of a policy Google announced on April 25, designed to boost “reliable sources” of information, after Google and other technology companies were criticized for allowing low-quality and even fraudulent websites to proliferate during the 2016 presidential campaign.

In a blog post, Ben Gomes, Google’s vice president of engineering, admitted that the company had been providing searchers with “offensive or clearly misleading content” in a small percentage of results. To combat this, Gomes wrote that Google had “adjusted [its] signals to help surface more authoritative pages and demote low-quality content.”

Gomes did not specify just what “adjusting signals” meant in regards to political content. Google and its employees have historically refused to release details about how results are filtered and prioritized . . .

“Not only are people being essentially blocked from accessing leftist viewpoints, they are not even being told they exist," he said. “We would think that the WSWS would be an authority on socialism, but apparently that’s not what Google thinks."

When asked how Bing, Google’s less-popular search rival, has handled the socialist publication, Damon said he had noted similar declines in referral traffic. “They’re just copying what Google did,” he said.”

Copied from: “Fake news” or free speech: Is Google cracking down on left media? - Salon.com - https://www.salon.com/2017/10/18/fake-news-or-free-speech-is-google-cracking-down-on-left-media/

Out of Google’s own mouth: (through Eric Schmidt, who from 2001 to 2011 served as the CEO of Google and who since then is the Executive Chairman of Google’s parent company, Alphabet):

“We’re very good at detecting what’s the most relevant and what’s the least relevant. It should be possible for computers to detect malicious, misleading and incorrect information and essentially have you not see it. We’re not arguing for censorship, we’re arguing just take it off the page, put it somewhere else…make it harder to find,” Schmidt said, adding: “I think we’re going to be ok.” (emphasis supplied)
Source: Alphabet’s Eric Schmidt on Google advertising and extremist content
Address : https://www.cnbc.com/2017/03/23/google-chairman-eric-schmidt-advertising-and-extremist-content.html

And this from the New York Times:
“In April, Google announced an initiative called Project Owl to provide “algorithmic updates to surface more authoritative content” and stamp out fake news stories from its search results.

To some, that was an uncomfortable step toward Google becoming an arbiter of what is and is not a trustworthy news source… . .

Most people have little understanding of how Google’s search engine ranks different sites, what it chooses to include or exclude, and how it picks the top results among hundreds of billions of pages. And Google tightly guards the mathematical equations behind it all — the rest of the world has to take their word that it is done in an unbiased manner.

Copied from: As Google Fights Fake News, Voices on the Margins Raise Alarm - The New York Times - https://www.nytimes.com/2017/09/26/technology/google-search-bias-claims.html?_r=0

Google however is NOT unbiased. It does have a political agenda OPPOSITE to those on the left.

For just one example, Google sees it as its mission to prevent the radicalization of a surplus population, again in Google’s own words (through Jared Cohen, director of Google Ideas):

"With more than 50 percent of the world’s population under the age of thirty and the vast majority of those characterized as “at risk” either socially, economically, or both, an oversupply exists of young people susceptible to recruitment by the extremist religious or ideological group closest to them in identity or proximity. Google Ideas and the Council on Foreign Relations aim to initiate a global conversation on how best to prevent young people from becoming radicalized and how to de-radicalize others.”
Quotes found at:
http://www.thinktankedblog.com/think-tanked/2011/03/cfr-jared-cohen-google-ideas-partner-counter-radicalization.html
and:
Google Ideas and Council on Foreign Relations Team-up on Counter-radicalization
https://www.cfr.org/blog/guest-post-google-ideas-and-council-foreign-relations-team-counter-radicalization

For more on Google’s evil politics, with other examples, see my backgrounder:

Creating an honest and unbiased alternative to Google Search must be a TOP PRIORITY of any left technological organization and May First needs to join and assist those already working on this. I’m not a techie myself, so I implore the techies in May First to work on saving the internet from this ongoing menace I’m addressing -and for its membership to vote to make this May First’s top priority. Here’s an article about techies already working on this:

“FREE SOFTWARE ACTIVISTS HAVE RELEASED A PEER-TO-PEER SEARCH ENGINE TO TAKE ON GOOGLE, YAHOO, BING AND OTHERS.

“The free, distributed search engine, YaCy, takes a new approach to search. Rather than using a central server, its search results come from a network of independent “peers,” users who have downloaded the YaCy software. The aim is that no single entity gets to decide what gets listed, or in which order results appear.

“Most of what we do on the Internet involves search. It’s the vital link between us and the information we’re looking for. For such an essential function, we cannot rely on a few large companies and compromise our privacy in the process,” said Michael Christen, YaCy’s project leader.

The project is supported by the Free Software Foundation Europe (FSFE), which is concerned that dominant search engines have too much control and power over what information Internet users can find online. “That company will also know what you’re currently interested in. The search terms used tell others a lot about what you’re up to. Targeted advertising is only the most benign use of this data,” explained Karsten Gerloff, FSFE president.

“We are moving away from the idea that services need to be centrally controlled. Instead, we are realizing how important it is to be independent, and to create infrastructure that doesn’t have a single point of failure,” added Gerloff.

The YaCy network currently has around 600 ‘peers’, but project organizers expect this to grow along the lines of other free software projects that aim to replace centrally-run services. For example, identi.ca (status.net) offers a free software alternative to Twitter; diaspora (joindiaspora.com) and many others provide a free, distributed alternative to Facebook.

As is often the case in the early stages of a new technology, results are better on some topics than on others – mainly computer-related issues.

The YaCy peers create individual search indexes and rankings, so that results better match what users are looking for over time. Each instance of the software contains a peer-to-peer network protocol to exchange search indexes with other YaCy search engines.

Everyone can try out the search engine at http://search.yacy.net/. Users can become part of YaCy’s network by installing the software on their own computers. YaCy is free software, so anyone can use, study, share and improve it. It is currently available for GNU/Linux, Windows and MacOS. The project is also looking for developers and other contributors.”

Source: Free Software Activists to Take on Google with New Free Search Engine | CIO
Address : https://www.cio.com/article/2401940/internet/free-software-activists-to-take-on-google-with-new-free-search-engine.html
(emphasis supplied).

On December 2nd, PLEASE VOTE TO MAKE ASSISTING WITH THE CREATION OF AN ALTERNATIVE TO CORPORATE CONTROLLED INTERNET SEARCH MAY FIRST’S TOP PRIORITY FOR 2018.

Thank you for your time and consideration!

From May First member, Angie D’Urso, website www.WhatNewsShouldBe.org

Anyone can contact me at angie.joy@verizon.net or Angie@WhatNewsShouldBe.org

2

This was super informative! I agree that an independent search engine is key.

What about DuckDuckGo? They’re open-source and use encryption. They also have a strong community of users who offer up “Instant Answers” which I’ve really enjoyed. I’m pretty sure they use their own algorithm because the results I get there do differ from Google and they promote that they don’t have a filter bubble like Google does.

3

Thanks. But DuckDuckGo is not independent - they use Microsoft’s Bing search engine. See for example:

“The Bing outage prevented DuckDuckGo, the privacy-centric engine that aggregates results from the Microsoft search engine, from returning useful links to queries”

Source: Outage outed: Bing dinged, Microsoft portal mortal, DuckDuckGo becomes DuckDuckNo • The Register
Address : https://www.theregister.co.uk/2017/08/15/bing_microsoft_duckduckgo_outage/

And as one of the articles I quoted from above noted:

"When asked how Bing, Google’s less-popular search rival, has handled the
socialist publication, Damon said he had noted similar declines in
referral traffic. “They’re just copying what Google did,” he said.”

Angie

1 Like
4

Totally agree with this use case, find the emphasis on corporate/institutional journalism worrying, and excited about the idea of a P2P alternative to host-model search infrastructures.

My main question would be — what, materially, do you propose that we do to assist in this project?

  • That we endorse YaCy?
  • That we all install instances of YaCy on our personal machines?
  • That we commit to educate folks about how install and use YaCy?
  • That we run an instance of YaCy as a public-facing server, so the general public can use this service?

Either way, looking forward to discussing.

1 Like
5

Dang, that’s too bad. Thanks for the clarification. I wonder if a campaign to get DuckDuckGo to adopt the YACy approach is feasible and worthwhile. I don’t know if that even makes sense, this realm of technology I’m pretty new to.

6

So about your main question, what do we do -

Note that the Yacy article I cited above was published on Nov. 28, 2011 - that means it’s 6 years old! So the first thing I think we should do is find out about the status of this project by having tech savvy people who understand this P2P language contact those behind it and find out - and if it’s still ongoing, ask them what they need and have them tell us all about it.

The second thing I suggest is that tech savvy people use their tech savvy newsgroups or whatever, connections, to find out what other alternatives to google are currently being attempted, and for each to discover what specific ills it’s supposed to cure, which are, or would be, controlled by what or who, for profit or not, etc. and their status, needs, chances of success, biggest obstacles to success, funding sources, strong textetc.? While eliminating this or that insufficient alternative along the way, like search engines whose only difference is they provide privacy to the searcher, but still deliver secret corporate-controlled results based on secret algorithms.

And remember, we’re not going to find out about these google alternatives easily by using google to find them. I wrote my first how-to-search-the-internet/Google-is-Evil post in July 2015 and when I was researching stuff for it, I was all about alternative search engines - yes, I was using google to do so - and didn’t find out about Yacy until earlier this Summer by accident! There are other Google substitutes also being attempted- here’s another one I found out about called “Find X”, open source, at https://www.findx.com/ - I’ll post some stuff about it below. I only found out about Find X recently in July through an email exchange I initiated to someone who had written something about google. So, techies, use your connections and specialized literature/websites/newsgroups/conferences/tweets whatever and find some report on the current state of alternatives to corporate-controlled internet search worldwide, with all the issues of concern addressed (of which there are so many), and if there isn’t a report already written that can be found, write one and let all May Firsters know about these possibilities as well as their own recommendation as to what kind of efforts they think we should make to help create ASAP an unbiased alternative to corporate-controlled internet search.

But after the techie experts find this all out, WE CANNOT JUST LEAVE IT IN THEIR HANDS & RECOMMENDATIONS (no offense :slight_smile:) as there are so many issues or variables to consider in an alternative search engine which are NOT technical issues, and need to be debated by one and all - like how we want the search engine to decide what to show us in the first few pages of our search results. Some things are easy and should be a given in any alternative search engine - like verbatim search - so whenever a searcher puts their query in quotation marks, websites which contain that exact phrase in quotes must appear first in the search results - something google doesn’t do anymore. Also boolean searches should be another given, unlike the google of today, and you must be able to search for anything by date or particular time frames.

After that, it gets tricky because you could have a search engine whose algorithm writers were not evil and whose political bent also matched our own (not like Google) still using algorithms that unintentionally provide biased results. Here’s an example. Should the search engine provide results first from the “most popular” sites discussing the search terms? And how do we want it to decide what’s most “popular”. Google uses “page rank” as one way to measure a website’s popularity - by counting how many other webpages have linked to a particular article - but about that, note this quote from my first google post:

“Google’s PageRank uses a popularity metric, treating inbound links
to a website as popularity votes, and votes from more popular sites are
weighted more than the lesser known ones. Such a metric favors majority over quality, often giving preference to those with financial power”
(For more quotes, see my 1st google post at http://tinyurl.com/Searching-Internet-Google-Evil )

So, how we want the search engine to find our stuff is not really a technical issue, only implementing chosen methods is. And if alternative search engines let people customize their search experience & use whichever options they prefer, we’d still need to ensure that those who have no time to look under the hood or consider how their search engine works, have the best non-corporate controlled system there is.

Now about Find X, some quotes from their site are copied below. BUT REMEMBER TO BE SKEPTICAL. The left was asleep at the switch during the Google metamorphosis from seemingly ‘not evil’ to definitely evil. Just because something at first blush appears to be good doesn’t make it so. (I HAVE NO IDEA WHY THE QUOTE BELOW IS IN SUCH LARGE BOLDED FONT. CAN’T FIGURE OUT HOW TO FIX IT.)

“Neutral search results
Unlike other private search engines, Findx is not a metasearch engine. We don’t rely on any of the big search engines - we do our own indexing and use our own data to provide search results. As a search engine valuing transparency, Findx is an open source project. We share as much of our source code as possible. (NOTE FROM ANGIE - SO THEY DON’T SHARE IT ALL. Probably the argument is that they have to protect against someone gaming their system and skewing results. But is there a tech way to avoid this, enabling all code to be shared?) We will also be implementing “algorithmic accountability” and create transparency about the works of our algorithms. In addition will let everyone adjust how our search algorithms should work for them.”

“Findx is a search engine based in Europe, one that respects your privacy and provides you with unbiased search results.As a European search engine, Findx fully complies with European laws and regulations, and is hosted in Europe.”

Source: About Findx | Privacore
Address : https://www.privacore.com/findx/about/#findxabout

To not be overwhelmed by this huge task, we should think of a search engine like a library. Make believe we were building the first library. How should it be designed? To get the most number of books into the smallest footprint, we can build really high bookshelves. But how can people reach up and see those books & how can we install those shelves to begin with? Techies/engineers - can you figure out some sort of ladder contraption for us? Now in the present day, the fact that so much can now be digitized and available virtually instantly at the touch of ones fingers through a phone or computer - text, video, audio, communications, books, etc., etc., etc. is one of the most amazing things ever - but ONLY if we can find the relevant & non-corporate-controlled from it. Such a noble, worthwhile goal for all of humanity. Information is always power. So thanks to all who use their thinking power on this project, and the researchers for their thinking and researching and the techies for their thinking and their teching . . . , and let’s make getting back control of the internet - by getting back control of internet search - our top priority.

Angie
One of my email addresses is NOT working (Angie@WhatNewsShouldBe.org)
so if you want to email me, please email either
Angie.joy@verizon.net or WhatNewsShouldBe@yahoo.com

7

I’m so glad this topic was brought up! I’ve been meaning to try out Yacy for some time, so I took a look and it was easy to download and get running - no fancy configuration, just install java and run it’s startup script. It’s looking promising… it started connecting to peers right away, and it looks like there were some bugs fixed this year, so it appears to be at least somewhat active, despite the age of the quoted article.

In terms of alternatives, I think it’s pretty clear that the standard centralized model is fundamentally flawed. The two glaring issues are censorship and privacy/survellance.

  1. Regarding censorship: in a centralized model, what gets displayed for which search terms is ultimately decided by the central authority/algorithm, whereas in an open source peer-to-peer model, that control is squarely in the hands of the people. Of course an implementation could potentially have some vulnerabilities that allow an attacker to somehow manipulate or influence the results. But that’s different from a built-in structural flaw that centralization has.

  2. Regarding privacy/survellance: you have to take DuckDuckGo or any privately held service on their word that they don’t hang on to an ip-to-search history database that they secretly sell to GoogleNSA. No matter how honest they sound when they say “we don’t track you”. Now, a distributed model alone doesn’t solve that - if the implementation involves broadcasting your search terms off to different peers, than those peers could just as well be on GoogleNSA servers. But again, that’s an implementation-specific issue. From what I can tell Yacy supports an option to not send terms out to peers, but instead search against the locally-stored index. (however, the index itself is distributed amongst the peers, so you couldn’t see results from the entire index) Also - just thinking out loud; dont’ know if Yacy has anything like this - I could imagine an implementation similar to how TOR uses layers of encryption to keep two peers talking to each other from ever knowing who the other one is…

Actually that makes me think of one concern I thought of… what if someone wanted to deliberately target someone running a peer on the network by sending search terms or site indexes relating to isis, pedo, etc. so that information resides on that persons hard drive, justifying a raid and guantanamo time? I don’t know if that’s a concern unique to this; there’s probably easier ways to accomplish the goal - heck, there’s probably already a specialized tool for it in vault 7.

Probably a more practical concern is that the actual matching algorithm is no where near as good as google. Meaning you need to sift through a lot of garbage in the results, so it’s considerably less convenient than google/duckduckgo/etc. That may make it difficult to get people to use it. Convenience is a problem with adoption of many alternatives… TOR web browser, or something like OpenStreetMaps and Nav-it as alternatives to google maps. (But not all google alternatives are like that - I’m finding NextCloud just as convenient as google docs/drive!)

Anyway, I’m not sure if Yacy is a panacea, but personally I’m convinced that its distributed model is probably the only real option for a free internet.

1 Like
8

I’m not sure what it means to make this a “top priority” but I don’t think our organization ever has absolute “top priorities” because of the type of organization we are. We never build priorities on work that only involves part of our membership and our “techies” right now comprise about two percent of May First’s membership.

Additionally, with Net Neutrality being destroyed and with the success of Tech and Rev, it seems that our set of priorities are going to be varied and diverse.

I wish we had more information about this new program YaCy. The people working on it aren’t the people we would expect to be working on such a project and its user base is, right now, very sparse. Obviously, the concept is potentially problematic as well because of the software download needed.

Still, I think we should explore it and review it along with other alternatives and see where we are going. That’s the LC’s job and membership should instruct LC to do just that.

My opinion. Thanks.

2 Likes
9

Cannot second this point enough.

In the app-ified landscape of modern consumer culture, having to download a piece of desktop software is a huge barrier to acquisition that would have to be overcome either by (very ambitious) users themselves, or be factored into the technology itself. Likewise, default settings (e.g. Bing as a default for Apple phones’ search) are our slipperiest problem.

Going back to this…

I think this might be our most simple and tangible contribution — giving people a Google alternative that they can use without an install process, and (the actual hard part) encouraging people to actually use it.

Thoughts?

10

Yea, agree — it’s going to take a lot of earnest energy to get this project off the ground.

11

Fighting to keep (or later restore) net neutrality can’t trump all else and certainly not efforts to develop non-corporate controlled internet search (and non-corporate controlled social media, etc.) If we’re successful and net neutrality wins then even while our political content can be retrieved at same internet speeds as corporate content, there will be nothing of our politics SEEN to retrieve at that same speed if corporations continue to control internet search and hide our stuff from appearing in search engines. Removing net neutrality is just one of a handful of tactics to cut us off from successfully using the internet towards our political goals of a better world and life for everyone. Corporate Controlled Internet Search is another tactic (gatekeeper of all information). And it gets short shrift on the left for no reason I can fathom while net neutrality gets lots of coverage and push for activist efforts. In any event, those like Alfredo who think this proposed May First 2018 priority about internet search isn’t greater than the one for net neutrality, doesn’t top it, will I assume concede that May First’s 2018 efforts and resources expended toward each of the 2 struggles, non-controlled internet search & net neutrality, should at least be equal. And what with the difficulties presented as expressed above in creating a valuable and workable way to search the internet without corporate control, we’ve got to get started already! The internet depends upon it.

Angie
Angie@WhatNewsShouldBe.org