Skip to content

Commit

Permalink
Merge branch 'develop'
Browse files Browse the repository at this point in the history
  • Loading branch information
@tlovett1
tlovett1 committed Jun 25, 2015
2 parents 2c25d6a + 9e446e0 commit 138a234
Show file tree
Hide file tree
Showing 16 changed files with 447 additions and 154 deletions.
19 changes: 13 additions & 6 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@ Integrate [Elasticsearch](http://www.elasticsearch.org/) with [WordPress](http:/

**Please note:** the master branch is the stable branch

**Latest stable release:** [1.4](https://github.com/10up/ElasticPress/releases/tag/v1.4)
**Latest stable release:** [1.4](https://github.com/10up/ElasticPress/releases/tag/1.4)

*Upgrade Notice: If you are upgrading to 1.4, new date features will not work until you re-index: wp elasticpress index --setup*

## Background

Expand Down Expand Up @@ -99,14 +101,15 @@ After running an index, ElasticPress integrates with `WP_Query` if and only if t
'tax_query' => array(
array(
'taxonomy' => 'taxonomy-name',
'terms' => array( ... ),
'field' => 'slug',
'terms' => array( 'term-slug-1', 'term-slug-2', ... ),
),
),
) );
```

```tax_query``` accepts an array of arrays where each inner array *only* supports ```taxonomy``` (string) and
```terms``` (string|array) parameters. ```terms``` is a slug, either in string or array form.
```tax_query``` accepts an array of arrays where each inner array *only* supports ```taxonomy``` (string), ```field``` (string), and
```terms``` (string|array) parameters. ```field``` must be set to ```slug``` and ```terms``` must be a string or array of term slug(s).

* The following shorthand parameters can be used for querying posts by specific dates:

Expand Down Expand Up @@ -384,6 +387,10 @@ The following are special parameters that are only supported by ElasticPress.
) );
```

* ```cache_results``` (*boolean*)

This is a built-in WordPress parameter that caches retrieved posts for later use. It also forces meta and terms to be pulled and cached for each cached post. It is extremely important to understand when you use this parameter with ElasticPress that terms and meta will be pulled from MySQL not Elasticsearch during caching. For this reason, ```cache_results``` defaults to false.

* ```sites``` (*int*/*string*/*array*)

This parameter only applies in a multi-site environment. It lets you search for posts on specific sites or across the network.
Expand Down Expand Up @@ -458,9 +465,9 @@ The following are special parameters that are only supported by ElasticPress.

The following commands are supported by ElasticPress:

* `wp elasticpress index [--setup] [--network-wide] [--posts-per-page] [--no-bulk]`
* `wp elasticpress index [--setup] [--network-wide] [--posts-per-page] [--no-bulk] [--offset]`

Index all posts in the current blog. `--network-wide` will force indexing on all the blogs in the network. `--setup` will clear the index first and re-send the put mapping. `--posts-per-page` let's you determine the amount of posts to be indexed per bulk index (or cycle). `--no-bulk` let's you disable bulk indexing.
Index all posts in the current blog. `--network-wide` will force indexing on all the blogs in the network. `--network-wide` takes an optional argument to limit the number of blogs to be indexed across where 0 is no limit. For example, `--network-wide=5` would limit indexing to only 5 blogs on the network. `--setup` will clear the index first and re-send the put mapping. `--posts-per-page` let's you determine the amount of posts to be indexed per bulk index (or cycle). `--no-bulk` let's you disable bulk indexing. `--offset` let's you skip the first n posts (don't forget to remove the `--setup` flag when resuming or the index will be emptied before starting again).

* `wp elasticpress activate`

Expand Down
79 changes: 51 additions & 28 deletions bin/wp-cli.php
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
<?php

if ( ! defined( 'ABSPATH' ) ) {
exit; // Exit if accessed directly.
}
WP_CLI::add_command( 'elasticpress', 'ElasticPress_CLI_Command' );

/**
Expand Down Expand Up @@ -34,8 +36,11 @@ class ElasticPress_CLI_Command extends WP_CLI_Command {
public function put_mapping( $args, $assoc_args ) {
$this->_connect_check();

if ( ! empty( $assoc_args['network-wide'] ) && is_multisite() ) {
$sites = ep_get_sites();
if ( isset( $assoc_args['network-wide'] ) && is_multisite() ) {
if ( ! is_numeric( $assoc_args['network-wide'] ) ){
$assoc_args['network-wide'] = 0;
}
$sites = ep_get_sites( $assoc_args['network-wide'] );

foreach ( $sites as $site ) {
switch_to_blog( $site['blog_id'] );
Expand Down Expand Up @@ -85,8 +90,11 @@ public function put_mapping( $args, $assoc_args ) {
public function delete_index( $args, $assoc_args ) {
$this->_connect_check();

if ( ! empty( $assoc_args['network-wide'] ) && is_multisite() ) {
$sites = ep_get_sites();
if ( isset( $assoc_args['network-wide'] ) && is_multisite() ) {
if ( ! is_numeric( $assoc_args['network-wide'] ) ){
$assoc_args['network-wide'] = 0;
}
$sites = ep_get_sites( $assoc_args['network-wide'] );

foreach ( $sites as $site ) {
switch_to_blog( $site['blog_id'] );
Expand Down Expand Up @@ -166,7 +174,7 @@ private function _create_network_alias() {
/**
* Index all posts for a site or network wide
*
* @synopsis [--setup] [--network-wide] [--posts-per-page] [--no-bulk]
* @synopsis [--setup] [--network-wide] [--posts-per-page] [--no-bulk] [--offset]
* @param array $args
*
* @since 0.1.2
Expand All @@ -182,8 +190,22 @@ public function index( $args, $assoc_args ) {
$assoc_args['posts-per-page'] = 350;
}

if ( ! empty( $assoc_args['offset'] ) ) {
$assoc_args['offset'] = absint( $assoc_args['offset'] );
} else {
$assoc_args['offset'] = 0;
}

$total_indexed = 0;

/**
* Prior to the index command invoking
* Useful for deregistering filters/actions that occur during a query request
*
* @since 1.4.1
*/
do_action( 'ep_wp_cli_pre_index', $args, $assoc_args );

// Deactivate our search integration
$this->deactivate();

Expand All @@ -196,16 +218,19 @@ public function index( $args, $assoc_args ) {
$this->put_mapping( $args, $assoc_args );
}

if ( ! empty( $assoc_args['network-wide'] ) && is_multisite() ) {
if ( isset( $assoc_args['network-wide'] ) && is_multisite() ) {
if ( ! is_numeric( $assoc_args['network-wide'] ) ){
$assoc_args['network-wide'] = 0;
}

WP_CLI::log( __( 'Indexing posts network-wide...', 'elasticpress' ) );

$sites = ep_get_sites();
$sites = ep_get_sites( $assoc_args['network-wide'] );

foreach ( $sites as $site ) {
switch_to_blog( $site['blog_id'] );

$result = $this->_index_helper( isset( $assoc_args['no-bulk'] ), $assoc_args['posts-per-page'] );
$result = $this->_index_helper( isset( $assoc_args['no-bulk'] ), $assoc_args['posts-per-page'], $assoc_args['offset'] );

$total_indexed += $result['synced'];

Expand All @@ -228,7 +253,7 @@ public function index( $args, $assoc_args ) {

WP_CLI::log( __( 'Indexing posts...', 'elasticpress' ) );

$result = $this->_index_helper( isset( $assoc_args['no-bulk'] ), $assoc_args['posts-per-page'] );
$result = $this->_index_helper( isset( $assoc_args['no-bulk'] ), $assoc_args['posts-per-page'], $assoc_args['offset'] );

WP_CLI::log( sprintf( __( 'Number of posts synced on site %d: %d', 'elasticpress' ), get_current_blog_id(), $result['synced'] ) );

Expand All @@ -250,15 +275,15 @@ public function index( $args, $assoc_args ) {
*
* @param bool $no_bulk disable bulk indexing
* @param int $posts_per_page
* @param int $offset
*
* @since 0.9
* @return array
*/
private function _index_helper( $no_bulk = false, $posts_per_page) {
private function _index_helper( $no_bulk = false, $posts_per_page, $offset = 0) {
global $wpdb, $wp_object_cache;
$synced = 0;
$errors = array();
$offset = 0;

while ( true ) {

Expand Down Expand Up @@ -299,16 +324,16 @@ private function _index_helper( $no_bulk = false, $posts_per_page) {
$offset += $posts_per_page;

usleep( 500 );

// Avoid running out of memory
$wpdb->queries = array();

if ( is_object( $wp_object_cache ) ) {
$wp_object_cache->group_ops = array();
$wp_object_cache->stats = array();
$wp_object_cache->memcache_debug = array();
$wp_object_cache->cache = array();

if ( is_callable( $wp_object_cache, '__remoteset' ) ) {
call_user_func( array( $wp_object_cache, '__remoteset' ) ); // important
}
Expand Down Expand Up @@ -427,23 +452,17 @@ private function bulk_index() {
*/
private function send_bulk_errors() {
if ( ! empty( $this->failed_posts ) ) {
$email_text = __( "The following posts failed to index:\r\n\r\n", 'elasticpress' );
$error_text = __( "The following posts failed to index:\r\n\r\n", 'elasticpress' );
foreach ( $this->failed_posts as $failed ) {
$failed_post = get_post( $failed );
if ( $failed_post ) {
$email_text .= "- {$failed}: " . get_post( $failed )->post_title . "\r\n";
$error_text .= "- {$failed}: " . $failed_post->post_title . "\r\n";
}
}
$send_mail = wp_mail( get_option( 'admin_email' ), wp_specialchars_decode( get_option( 'blogname' ) ) . __( ': ElasticPress Index Errors', 'elasticpress' ), $email_text );

if ( ! $send_mail ) {
fwrite( STDOUT, __( 'Failed to send bulk error email. Print on screen? [y/n] ' ) );
$answer = trim( fgets( STDIN ) );
if ( 'y' == $answer ) {
WP_CLI::log( $email_text );
}
}
// clear failed posts after sending emails
WP_CLI::log( $error_text );

// clear failed posts after printing to the screen
$this->failed_posts = array();
}
}
Expand All @@ -456,7 +475,9 @@ private function send_bulk_errors() {
public function status() {
$this->_connect_check();

$request = wp_remote_get( trailingslashit( EP_HOST ) . '_status/?pretty' );
$request_args = array( 'headers' => ep_format_request_headers() );

$request = wp_remote_get( trailingslashit( EP_HOST ) . '_status/?pretty', $request_args );

if ( is_wp_error( $request ) ) {
WP_CLI::error( implode( "\n", $request->get_error_messages() ) );
Expand All @@ -477,7 +498,9 @@ public function status() {
public function stats() {
$this->_connect_check();

$request = wp_remote_get( trailingslashit( EP_HOST ) . '_stats/' );
$request_args = array( 'headers' => ep_format_request_headers() );

$request = wp_remote_get( trailingslashit( EP_HOST ) . '_stats/', $request_args );
if ( is_wp_error( $request ) ) {
WP_CLI::error( implode( "\n", $request->get_error_messages() ) );
}
Expand Down
Loading

0 comments on commit 138a234

Please sign in to comment.