The idea of this app is to provide something that can edit a
photo and apply Machine Learning algorithms to help 'fix' the
photo. It will also provide more standard features such as
allowing the user to manually adjust exposure etc. and apply
various filters and transforms.
This is essentially a port of the philter app to Apple's Metal
framework (instead of OpenGLES), with Machine Learning
algorithms added for the 'auto' functions.
Note: Info on the current page relates to approaches that might
be usable in this app. For info purely related to implementing
Machine Learning on iOS, check out the Machine Learning page.
In general, the app will provide the following types of
functions (not including 'auto' functionality, which I'll list
separately):
The intent is to review academic and open source Machine
Learning (ML) papers and algorithms that provide various 'smart'
or 'automatic' image adjustments or transformations. I will then
port those algorithms to Keras (a Python-based ML framework that
sits above Tensorflow or Theros) and verify on a GPU-enabled AWS
server. Once verified, I will then port the Keras model to a
Metal model, which can then be used on an iOS platform. Another
approach would be to build TensorFlow into the app and call
TensorFlwo directly.
The reason for the many steps is that the ML algorithms require
a training (and verification) phase which is computationally
very expensive, and best done on high end platforms that are
easily available from Amazon (for example). Also, it is much
easier to port the general training algorithm to a Python-based
environment like Keras than it would be to port it directly to
iOS, plus you can use the many Python-based libraries and tools
to process the data and tune the algorithm. Once trained, the
model is relatively easy to use in an app. BTW, the reason I
chose Keras is that the iOS Metal framework is able to process
Keras models.
The types of functions that I plan to implement include:
So, there are lots of algorithms out there that implement various forms of "auto-fixing" photos. The ones that seem most promising to me include:
BTW, the image dataset is available for download, take a look at:
and here is a list of some potentially useful algorithms related to this problem space:
This is an operation something like Photoshop's Auto White
Balance function, i.e. automatically adjusting the histogram to
eliminate colour casts.
There are several ways to do this:
Here are some random notes and links related to implementation approaches and procedures