Windows 10: Snip Insights brings AI to Snipping tool for Windows 10

  1. Brink's Avatar
    Posts : 33,086
    64-bit Windows 10 Pro build 18262
       10 May 2018 #1

    Snip Insights brings AI to Snipping tool for Windows 10

    What if everyday work flows were re-imagined through the power of AI to inspire and delight? What if users could achieve more with a tool that is both easily accessible and familiar?Millions of Windows users perform the same task every single day. Screenshots are taken all the time, but as technologies progress in leaps and bounds, screen capturing tools haven’t really changed at all.

    Snip Insights is a Microsoft Garage intern project focused on injecting AI into the screen capturing process.

    The project has three primary goals. One is to revolutionize the way you search, shifting the paradigm from text-based search to image-based search. Another is to increase productivity by providing insights just a snip away. The last is generate insights about the limitless possibilities of AI.

    About the team
    At Microsoft Vancouver, the curiosities of 7 interns were piqued by a pitch from the Cloud AI team. The intern team, composed of 5 developers, 1 designer, and 1 program manager, was challenged to re-imagine widely used tools that millions of users love through the capabilities of AI. Thus, Snip Insights was born.

    Snip Insights team (l to r) David Wu, Mehdi Chaid, Christie Chan, Paulo Aguiar Araujo, coach (on-screen Evgeny Vinnik, coach), Athi Narayanan Jeyaraj, Jeffrey Chang (on-screen Marius Ungureanu, coach), Vivy Wang, Charmaine Lee

    What is Snip Insights?
    Snip Insights is an open source desktop utility that enables Windows users to retrieve image intelligent insights through a simple screenshot. Screenshots are essentially snapshots of peoples’ lives. Snip Insights helps you achieve more with those stories by leveraging Cloud AI services to convert images to translated text, automatically detect and tag your image content, along with many more smart image suggestions that would improve your workflows while showcasing Azure Cognitive Services’ potential.

    Imagine that you have a scan of a textbook or work report. Rather than having to manually type out the information, snipping it will return editable text in just one click. Or maybe you’re scrolling through your social media feed and come across someone wearing a cool pair of shoes, you can simply snip to find out where to purchase them! Snip Insights can show you relevant information based on what you’ve just snipped – including identifying famous people, places, or landmarks.

    Building Snip Insights
    Injecting the capabilities of AI seemed technically daunting. Finding focus was also a painstaking trial. The intern team was given an arsenal of resources, but they had to forge the path to a product on their own through endless sessions of brainstorming, whiteboarding, and mind mapping.

    From a technical standpoint, it came down to building off the Windows Snipping Tool or Snip, a retired Microsoft Garage project that let users annotate a photo or screenshot with voice and ink and share with others.. Ultimately, the team chose Snip for a more straightforward integration of Azure Cognitive Services and for overall ease of releasing as an independent tool. Snip, built on the WPF platform using C#, was a solid foundation for this project.

    Quick iterations
    An integral part of the experience was conducting user research and user-focused development. The team constantly tested out iterations internally, working with developers, designers, PMs, and stakeholders. It became clear that one of the biggest challenges would be to decide on a target audience.

    Any computer user has access to screen capturing tools. Feedback from users varied drastically; each person used her or his tools in diverse ways and for different purposes. The interns figured out a lot of things just by trial and error. They learned to react quickly to change, whether that be a newly discovered visual design problem or a sudden technical obstacle. At one point, just a few days before a key milestone, the team decided to entirely redesign the user interface based on critical user feedback, to further distance the product from the original Snip visual design and bridge the gap to the Fluent design language. The ability to iterate quickly as a team proved to be the most invaluable skill.

    Endless possibilities of community-driven development
    By releasing the project open-source, the team hopes to inspire developers around the world to innovate and improve on their efforts. The Snip Insights GitHub repository will be maintained by the Cloud AI team. The internship was just a short four months where the interns worked on their own team, developing and iterating on this project as part of The Garage Internship Program in Vancouver. Though they were chartered by the Cloud AI group with a specific challenge, the solution possibilities were many. There were various use cases explored but not realized. On such use-case was to inject intelligence into the actual snipping process. What if the tool were smart enough to recognize object edges and shapes, cutting out additional steps and third-party software completely?

    With the power of Microsoft AI and Cognitive Services, any developer can develop intelligent applications that will be loved by users around the world. If you have the motivation, curiosity, and drive, all the information and resources are openly available.

    Snip Insights is now on GitHub and this is an invitation for you to contribute your ideas!

    Source: Snip Insights brings AI to Snipping tool with intelligent insights - Microsoft Garage

    Available at: GitHub - Microsoft/Snip-Insights: Snip Insights, a Microsoft Garage project
    Last edited by Brink; 11 May 2018 at 12:12.
      My ComputersSystem Spec


Related Threads
Hello, Just upgraded to Win 10 from Win 7. I am trying to like it :). One of the annoyances I've found so far is the snipping tool. When on Win 7, when I did my shortcut keys to open it, it would automatically go into snipping mode so I...
So ive been able to find a means to add the snipping tool itself to my context menu. Ive also been able to find a means to add a snipping tool entry with cascading menu to my start menu. I'm looking specifically for a means to add the snipping...
Snipping Tool under Windows 10 in Software and Apps
I've been using Windows Snipping Tool since Windows 7 to quickly capture screenshots of parts of the screen. But now on Windows 10 it doesn't work the same way. Say, when I click its icon, on Windows 7 the Snipping Tool would let me select the...
Solved Snipping Tool in Windows 10 in Software and Apps
I have an old shortcut to the Snipping Tool and just clicked on it after upgrade. It launches but doesn't seem to be able to save anything any longer. Anyone knows anything about this?
Can I use the Snip Tool to Snip a Still Frame from a You Tube Video? If so how do I do this at full screen without the bottom Task Bar getting in the way?
Our Sites
Site Links
About Us
Windows 10 Forums is an independent web site and has not been authorized, sponsored, or otherwise approved by Microsoft Corporation. "Windows 10" and related materials are trademarks of Microsoft Corp.

© Designer Media Ltd
All times are GMT -5. The time now is 02:26.
Find Us