I really wanted to study Ruby better, but there was no working draft. And I tried to write a gem to work with the Yandex Direct API.

There were several reasons. Among them: Yandex Direct API is very typical for Yandex and modern REST services in general. If you understand and overcome typical errors, you can easily and quickly write counterparts for other Yandex APIs (and not only). And one more thing: all the analogs that I managed to find had problems with the support of the Yandex.Direct versions: some were sharpened for 4, others for the new 5, and I never found support for units.

Metaprogramming is a great thing.

The main idea of ​​gem-a - once in a language like Ruby or Python, you can create new methods and JSON-like objects on the fly, the interface methods for accessing the REST service can repeat the functions of the Rest-service itself. To be able to write like this:

request = { "SelectionCriteria" => { "Types" => ["TEXT_CAMPAIGN"] }, "FieldNames" => ["Id", "Name"], "TextCampaignFieldNames" => ["BiddingStrategy"] } options = { token: Token } @direct = Ya::API::Direct::Client.new(options) json = direct.campaigns.get(request) 

And instead of writing help, send users to the manuals on the specified API.

Methods from old versions call, for example, like this:

 json = direct.v4.GetCampaignsList 

In case you are not interested in reading, but want to try - you can get a ready-made gem from here:

• ya-api-direct on RubyGems
• ya-api-direct on github

You can learn about getting an omniauth-token from rails from the twitter example . And the names of the methods and the registration procedure are described in great detail in the documentation from Yandex .

If details are interesting - they are further.

We start the development

Of course, the article describes the most basic experience and the simplest things. But it can be useful for beginners (like me), as a reminder on creating a typical gem. It is interesting to collect information on articles, of course, but for a long time.

Finally, it may be that some of the readers really need to quickly add support for the Yandex Direct API to their project.

And also it will be useful to me - in terms of feedback.

Check script

To begin, let's register in Yandex Direct, create a test application there and get a temporary Token for it.

Then open the Yandex Direct API help and learn how to call methods. Somehow:

For version 5:

 require "net/http" require "openssl" require "json" Token = "TOKEN" #    TOKEN. def send_api_request_v5(request_data) url = "https://%{api}.direct.yandex.com/json/v5/%{service}" % request_data uri = URI.parse(url) request = Net::HTTP::Post.new(uri.path, initheader = { 'Client-Login' => request_data[:login], 'Accept-Language' => "ru", 'Authorization' => "Bearer #{Token}" }) request.body = { "method" => request_data[:method], "params" => request_data[:params] }.to_json http = Net::HTTP.new(uri.host, uri.port) http.use_ssl = true http.verify_mode = OpenSSL::SSL::VERIFY_NONE response = http.request(request) if response.kind_of? Net::HTTPSuccess JSON.parse response.body else raise response.inspect end end p send_api_request_v5 api: "api-sandbox", login: "YOUR LOGIN HERE", service: "campaigns", method: "get", params: { "SelectionCriteria" => { "Types" => ["TEXT_CAMPAIGN"] }, "FieldNames" => ["Id", "Name"], "TextCampaignFieldNames" => ["BiddingStrategy"] } 

For version 4 Live (Token fits both):

 require "net/http" require "openssl" require "json" Token = "TOKEN" #    TOKEN. def send_api_request_v4(request_data) url = "https://%{api}.direct.yandex.com/%{version}/json/" % request_data uri = URI.parse(url) request = Net::HTTP::Post.new(uri.path) request.body = { "method" => request_data[:method], "param" => request_data[:params], "locale" => "ru", "token" => Token }.to_json http = Net::HTTP.new(uri.host, uri.port) http.use_ssl = true http.verify_mode = OpenSSL::SSL::VERIFY_NONE response = http.request(request) if response.kind_of? Net::HTTPSuccess JSON.parse(response.body) else raise response.inspect end end p send_api_request_v4 api: "api-sandbox", version: "live/v4", method: "GetCampaignsList", params: [] 

These scripts are already suitable for debugging and quick test requests.

But, as the (mythical) person-month teaches us, the script for itself and the library for others are two different classes of applications. And to transfer one to the other, to sweat.

Create a gem

For a start it was necessary to determine the name - simple and not busy. And he came to the conclusion that ya-api-direct is what is needed.

Firstly, the structure itself is logical - and if, for example, ya-api-weather appears, it will be clear what it refers to. Secondly, I still do not have an official product from Yandex to use the trademark as a prefix. In addition, it is a hint of ya.ru , where the old laconic design is carefully kept.

It’s a bit lazy to create all the folders with your hands. Let the bundler do it for us:

 bundle gem ya-api-direct 

As a tool for UnitTest, I have indicated a minitest. Then it will be clear why.

Now we have a folder, and in it is a gem ready for assembly. Its only flaw is that it is completely empty.

But now we fix it.

We write tests

UnitTests are incredibly useful for detecting slyly hidden bugs. Almost every programmer, who nevertheless was able to write them and fix a couple of dozen bugs that hid in the source, promises himself that he will always write them now. But still does not write.

In some projects (probably, they are written by especially non-lazy programmers) there are both test and spec-tests. But in the latest versions of minitest I suddenly learned the spec interface, and I decided to get along with some specs.

Since we have an online interface and, moreover, points are written off for each request, we will fake the answers from Yandex Direct API. For this we need a tricky gem webmock .

Add to gems

 group :test do gem 'rspec', '>= 2.14' gem 'rubocop', '>= 0.37' gem 'webmock' end 

Update, rename the test folder to spec. Since I was in a hurry, I wrote tests for external interfaces only.

 require 'ya/api/direct' require 'minitest/autorun' require 'webmock/minitest' describe Ya::API::Direct::Client do Token = "TOKEN" #  , .. API   . before do @units = { just_used: 10, units_left: 20828, units_limit: 64000 } units_header = {"Units" => "%{just_used}/%{units_left}/%{units_limit}" % @units } @campaigns_get_body = { #     Yandex Direct API    } #    stub_request(:post, "https://api-sandbox.direct.yandex.ru/json/v5/campaigns") .with( headers: {'Accept'=>'*/*', 'Accept-Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3', 'Accept-Language'=>'en', 'Authorization'=>'Bearer TOKEN', 'Client-Login'=>'', 'User-Agent'=>'Ruby'}, body: {"method" => "get", "params"=> {}}.to_json) .to_return(:status => 200, body: @campaigns_get_body.to_json, headers: units_header) #     @clientV4 = Ya::API::Direct::Client.new(token: Token, api: :v4) @clientV5 = Ya::API::Direct::Client.new(token: Token) end 

webmock replaces the methods of standard libraries for working with HTTP, so that for requests with a specific body and headers the corresponding response is returned.

If you make a mistake setting, it's not scary. When you try to send a request that is not in the filter, the webmock will report an error and even tell you how to write the stub correctly.

And we write specs:

 describe "when does a request" do it "works well with version 4" do assert @clientV4.v4.GetCampaignsList == @campaigns_get_body end it "works well with version 5" do assert @clientV5.campaigns.get == @campaigns_get_body end end #    

Rake

Rake is implemented so flexibly and simply that almost in every library it has its own structure. So I just told him to run all the files that are called spec _ *. Rb and are in the spec directory:

 require "bundler/gem_tasks" require "rake/testtask" task :spec do Dir.glob('./spec/**/spec_*.rb').each { |file| require file} end task test: [:spec] task default: [:spec] 

Now our spec can be called like this:

 rake test 

Or even:

 rake 

True, he has nothing to test.

Write gem

First, fill in with information about gem (without this bundle will refuse to run). Then we write to gemspec which third-party libraries we will use.

 gem 'jruby-openssl', platforms: :jruby gem 'rake' gem 'yard' group :test do gem 'rspec', '>= 2.14' gem 'rubocop', '>= 0.37' gem 'webmock' gem 'yardstick' end 

Do

 bundle install 

and go to lib to create files.

We will have the following files:

• client.rb - external interface
• direct_service_base.rb - basic service for working with API
• direct_service_v4.rb - service for working with API 4 and 4 Live
• direct_service_v5.rb - service for working with API 5
• gateway.rb - forwards and processes network requests =
• url_helper.rb - all sorts of static functions that are not in the gateway.rb
• constants.rb - list of available methods Yandex DIrect API
• exception.rb - an exception to show API errors
• version.rb - service file with version settings

Controllers for different versions

To begin with, we will create a file with constants, in which we will write all the functions from the API.

contants.rb

 module Ya module API module Direct API_V5 = { "Campaigns" => [ "add", "update", "delete", "suspend", "resume", "archive", "unarchive", "get" ], #  .. } API_V4 = [ "GetBalance", #  .. ] API_V4_LIVE = [ "CreateOrUpdateCampaign", #  .. ] end end end 

Now we will create a basic service wrapper, from which we will inherit the service for versions 4 and 5.

direct_service_base.rb

 module Ya::API::Direct class DirectServiceBase attr_reader :method_items, :version def initialize(client, methods_data) @client = client @method_items = methods_data init_methods end protected def init_methods @method_items.each do |method| self.class.send :define_method, method do |params = {}| result = exec_request(method, params || {}) callback_by_result result result[:data] end end end def exec_request(method, request_body) client.gateway.request method, request_body, @version end def callback_by_result(result={}) end end end 

In the constructor, it gets the source client and the list of methods. And then creates them within itself through: define_method.

And why can't we do with the method respond_to_missing? (as many gems still do)? Because it is slower and not so convenient. And without that slow interpreter gets into it after an exception and checking in is_respond_to_missing? .. In addition, the methods created in this way fall into the results of the methods call, which is convenient for debugging.

Now we will create a service for versions 4 and 4 Live.

direct_service_v4.rb

 require "ya/api/direct/constants" require "ya/api/direct/direct_service_base" module Ya::API::Direct class DirectServiceV4 < DirectServiceBase def initialize(client, methods_data, version = :v4) super(client, methods_data) @version = version end def exec_request(method, request_body = {}) @client.gateway.request method, request_body, nil, (API_V4_LIVE.include?(method) ? :v4live : @version) end end end 

In version 5, the server not only responds to user requests, but also reports how many points were spent on the last request, how many were left and how many were in the current session. Our service should be able to disassemble them (but we have not yet written how he will do it). But we will indicate in advance that he must update the fields in the main client class.

direct_service_v5.rb

 require "ya/api/direct/direct_service_base" module Ya::API::Direct class DirectServiceV5 < DirectServiceBase attr_reader :service, :service_url def initialize(client, service, methods_data) super(client, methods_data) @service = service @service_url = service.downcase @version = :v5 end def exec_request(method, request_body={}) @client.gateway.request method, request_body, @service_url, @version end def callback_by_result(result={}) if result.has_key? :units_data @client.update_units_data result[:units_data] end end end end 

By the way, have you noticed that some mysterious gateway is responsible for calling the request?

Gateway and UrlHelper

The Gateway class provides queries. It moved most of the code from our script.
gateway.rb

 require "net/http" require "openssl" require "json" require "ya/api/direct/constants" require "ya/api/direct/url_helper" module Ya::API::Direct class Gateway #    def request(method, params, service = "", version = nil) ver = version || (service.nil? ? :v4 : :v5) url = UrlHelper.direct_api_url @config[:mode], ver, service header = generate_header ver body = generate_body method, params, ver uri = URI.parse url request = Net::HTTP::Post.new(uri.path, initheader = header) request.body = body.to_json http = Net::HTTP.new(uri.host, uri.port) http.use_ssl = true http.verify_mode = @config[:ssl] ? OpenSSL::SSL::VERIFY_PEER : OpenSSL::SSL::VERIFY_NONE response = http.request(request) if response.kind_of? Net::HTTPSuccess UrlHelper.parse_data response, ver else raise response.inspect end end #     generate_header  generate_body #    ,   end end 

The standard Net :: HTTP is involved because it is simple as a rake. Quite possible to send requests from faraday . OmniAuth (about which I will discuss below) works on it, and so the application will not overgrow any extra gems.

Finally, UrlHelper is filled with static functions that generate URLs, parse data and parses Units (which is easy):

 require "json" require "ya/api/direct/exception" module Ya::API::Direct RegExUnits = Regexp.new /(\d+)\/(\d+)\/(\d+)/ class UrlHelper def self.direct_api_url(mode = :sandbox, version = :v5, service = "") format = :json protocol = "https" api_prefixes = { sandbox: "api-sandbox", production: "api" } api_prefix = api_prefixes[mode || :sandbox] site = "%{api}.direct.yandex.ru" % {api: api_prefix} api_urls = { v4: { json: '%{protocol}://%{site}/v4/json', soap: '%{protocol}://%{site}/v4/soap', wsdl: '%{protocol}://%{site}/v4/wsdl', }, v4live: { json: '%{protocol}://%{site}/live/v4/json', soap: '%{protocol}://%{site}/live/v4/soap', wsdl: '%{protocol}://%{site}/live/v4/wsdl', }, v5: { json: '%{protocol}://%{site}/json/v5/%{service}', soap: '%{protocol}://%{site}/v5/%{service}', wsdl: '%{protocol}://%{site}/v5/%{service}?wsdl', } } api_urls[version][format] % { protocol: protocol, site: site, service: service } end def self.extract_response_units(response_header) matched = RegExUnits.match response_header["Units"] matched.nil? ? {} : { just_used: matched[1].to_i, units_left: matched[2].to_i, units_limit: matched[3].to_i } end private def self.parse_data(response, ver) response_body = JSON.parse(response.body) validate_response! response_body result = { data: response_body } if [:v5].include? ver result.merge!({ units_data: self.extract_response_units(response) }) end result end def self.validate_response!(response_body) if response_body.has_key? 'error' response_error = response_body['error'] raise Exception.new(response_error['error_detail'], response_error['error_string'], response_error['error_code']) end end end end 

If the server returned an error, we throw an Exception with its text.

The code looks self-evident and that is quite good. Self-explanatory code is easier to maintain.

Client

Now we need to write the client class itself, with which the external interfaces interact. Since most of the functionality has already moved to the inner classes, it will be very short.

 require "ya/api/direct/constants" require "ya/api/direct/gateway" require "ya/api/direct/direct_service_v4" require "ya/api/direct/direct_service_v5" require "ya/api/direct/exception" require 'time' module Ya::API::Direct AllowedAPIVersions = [:v5, :v4] class Client attr_reader :cache_timestamp, :units_data, :gateway, :v4, :v5 def initialize(config = {}) @config = { token: nil, app_id: nil, login: '', locale: 'en', mode: :sandbox, format: :json, cache: true, api: :v5, ssl: true }.merge(config) @units_data = { just_used: nil, units_left: nil, units_limit: nil } raise "Token can't be empty" if @config[:token].nil? raise "Allowed Yandex Direct API versions are #{AllowedVersions}" unless AllowedAPIVersions.include? @config[:api] @gateway = Ya::API::Direct::Gateway.new @config init_v4 init_v5 start_cache! if @config[:cache] yield self if block_given? end def update_units_data(units_data = {}) @units_data.merge! units_data end def start_cache! case @config[:api] when :v4 result = @gateway.request("GetChanges", {}, nil, :v4live) timestamp = result[:data]['data']['Timestamp'] when :v5 result = @gateway.request("checkDictionaries", {}, "changes", :v5) timestamp = result[:data]['result']['Timestamp'] update_units_data result[:units_data] end @cache_timestamp = Time.parse(timestamp) @cache_timestamp end private def init_v4 @v4 = DirectServiceV4.new self, (API_V4 + API_V4_LIVE) end def init_v5 @v5 = {} API_V5.each do |service, methods| service_item = DirectServiceV5.new(self, service, methods) service_key = service_item.service_url @v5[service_key] = service_item self.class.send :define_method, service_key do @v5[service_key] end end end end end 

Methods of version 4 are written into the property v4, methods of version 5, grouped by individual services, become methods of the client class through the construction already familiar to us. Now, when we call client.campaigns.get, Ruby will first execute client.campaigns (), and then call the get method on the received service.

The last term of the constructor is needed so that the class can be used in the do ... end construction.

Immediately after initialization, it executes (if it is specified in the settings) start_cache! To send an API command to enable caching. The version in the settings only affects this, you can call the methods of both versions from the class instance. The resulting date will be available in the cache_timestamp property.

And in the units_data property will be the latest information on Units.

Also in the project there is a class with version settings and exceptions. With them, everything is so clear that even nothing to say. The class with the version settings is generated by the bundle along with the project.

Well, the file direct.rb need to specify those classes that should be visible to the user from the outside. In our case, this is only the client class. Plus version and exception (they are completely official).

Compile and fill

To compile the gem, you can follow the manual with RubyGems.org (there is nothing complicated). Or use the Mountable Engine from Rails .

And then we load on rubygems - suddenly this gem can be useful not only to us.

How to get token from Ruby on Rails

Logging in from Rails to the Yandec API and getting a token is very easy for any developer ... if not for the first time.

As we already learned, a token is required to access the Direct API. From the help from Yandex, it follows that we have before us - the good old OAuth2, which is used by a bunch of services, including Twitter and Facebook.

For Ruby, there is a classic gem omniauth , from which they inherit OAuth2 implementations for various services. Omniauth-yandex is already implemented. With him, we will try to figure it out.

Create a new rails application (we’ll add to work projects after we learn). Add to Gemfile:

 gem "omniauth-yandex" 

And do bundle install.

And then we use any manual for installing Omniauth authentication for rails. Here is an example for twitter . I think it’s not worth translating and retelling it - the article turned out to be so huge.

I have an example described in the article earned. The only amendment was that I did not write additional indexes to the User table, because SQLite does not support them.

True, the article does not indicate where the token is hidden. But it is not a secret at all. In SessionController it will be available through

  request.env['omniauth.auth'].credentials.token 

Just do not forget - each such authentication generates a token again. And if you then try to use scripts with a direct indication of token, the server will say that the old one is no longer suitable. You must return to the settings of the Yandex application, specify the debug callback URL again (__ https://oauth.yandex.ru/verification_code__ ), and then regenerate the token.

And even better, create a separate application for a static token so that debugging does not interfere.

Links

• ya-api-direct on RubyGems

• ya-api-direct on github

  
  
• Yandex OAuth Help
• Help on Yandex Direct API
• Online course on Yandex Direct API

• Omniauth-twitter setup

  
  
• omniauth-yandex
• webmock